Notation: $\{x\} = x - \lfloor x \rfloor$ Note that $$\left \lfloor \frac{pk}{q} \right\rfloor^2 = \left(\frac{pk}{q} - \left\{ \frac{pk}{q} \right\} \right)^2 = \left(\frac{pk}{q} \right)^2 - 2\cdot \frac{pk}{q} \left\{ \frac{pk}{q} \right\} + \left\{ \frac{pk}{q} \right\}^2 = -\left(\frac{pk}{q} \right)^2 + 2 \cdot \frac{pk}{q} \left \lfloor \frac{pk}{q} \right\rfloor + \left\{ \frac{pk}{q} \right\}^2 $$Since $\gcd(p,q)=1$, we have $\left\{\left\{\frac{pk}{q} \right\} : 1\leq k\leq q-1\right\} = \left\{\frac{k}{q} : 1\leq k\leq q-1\right\}$. Hence, \begin{align*} \sum_{k=1}^{q-1} \left \lfloor \frac{pk}{q} \right\rfloor^2 &= \frac{2p}{q} \sum_{k=1}^{q-1} k \left\lfloor \frac{pk}{q} \right\rfloor -\sum_{k=1}^{q-1} \left(\frac{pk}{q}\right)^2 +\sum_{k=1}^{q-1} \left(\frac{k}{q} \right)^2 \\ &= \frac{2p}{q} \sum_{k=1}^{q-1} k \left\lfloor \frac{pk}{q} \right\rfloor - \frac{(p^2-1)(q-1)(2q-1)}{6q} \end{align*}Since $\gcd(p,6)=1\implies 6|p^2-1$, $$\sum_{k=1}^{q-1} \left \lfloor \frac{pk}{q} \right\rfloor^2 \equiv q \sum_{k=1}^{q-1} \left \lfloor \frac{pk}{q} \right\rfloor^2 = 2p \sum_{k=1}^{q-1} k \left \lfloor \frac{pk}{q} \right\rfloor - \frac{p^2-1}{6}\cdot (q-1)(2q-1) \equiv 2p \sum_{k=1}^{q-1} k \left \lfloor \frac{pk}{q} \right\rfloor (\text{mod} q-1) $$