Nice inequality. To make things clear, rewrite the expression as $$ \sum_{1 \le i <j \le n} ( 2\lfloor x_ix_j \rfloor - \lfloor x_i^2 \rfloor-\lfloor x_j^2 \rfloor)$$Now it boils down to estimate $2\lfloor ab \rfloor - \lfloor a^2 \rfloor-\lfloor b^2 \rfloor$. Trivial bound and AM-GM give that $$ 2\lfloor ab \rfloor \le 2ab \le a^2+b^2 <\lfloor a^2 \rfloor+ \lfloor b^2 \rfloor+2$$Since both sides are integers, we deduce that $ 2\lfloor ab \rfloor \le \lfloor a^2 \rfloor+ \lfloor b^2 \rfloor+1$. Note that $2\lfloor ab \rfloor$ is even, the equality could hold, only when $\lfloor a^2 \rfloor$ and $\lfloor b^2 \rfloor$ have different parity; Otherwise, $ 2\lfloor ab \rfloor \le \lfloor a^2 \rfloor+ \lfloor b^2 \rfloor$. So if there are $t$ odd integers among $\lfloor x_i^2 \rfloor $, then at most $t(n-t)$ terms in the sum could be $1$, i.e. $$ \sum_{1 \le i <j \le n} ( 2\lfloor x_ix_j \rfloor - \lfloor x_i^2 \rfloor-\lfloor x_j^2 \rfloor) \le t(n-t) \le \lfloor \frac{n^2}{4} \rfloor$$ The equality case varies: let $x_1=x_2=\cdots=x_{\lfloor n/2 \rfloor}=a$, $x_{\lfloor n/2 \rfloor +1}=\cdots=x_n=b$, where $a,b$ satisfy that $ 2\lfloor ab \rfloor = \lfloor a^2 \rfloor+ \lfloor b^2 \rfloor+1$. For example, $a=0.9,b=1.2$ works well.

I did not see the nice idea in terms of $\lfloor a^2\rfloor$ being even or odd. But here is a different solution which does not seem more complicated: Let $f(n)$ be the desired maximum. Finding the maximal example is easy, so the whole issue is about an upper bound for $f(n)$. Rewriting the quantity as \[\sum_{1 \le i <j \le n} ( 2\lfloor x_ix_j \rfloor - \lfloor x_i^2 \rfloor-\lfloor x_j^2 \rfloor)\]one easily sees that $\frac{f(n)}{\binom{n}{2}} \le \frac{f(k)}{\binom{k}{2}}$ whenever $k \le n$, by averaging over all $k$-tuples $\{i_1,i_2,\dots,i_k\} \subset \{1,2,\dots,n\}$. In particular, we have $f(n) \le \frac{n}{n-2} \cdot f(n-1)$. From here one can easily prove that $f(n) \le \frac{n^2}{4}$ by induction once we have it for $n=3$. But we clearly have $f(2)=1$ and our recursion shows that $f(3) \le 3$. However, $f(3)$ is clearly even (since the expression is always even for $n$ odd!), hence $f(3)=2$ and everything goes through as desired.