Tintarn wrote: Positive integers $a$, $b$ and $c$ satisfy the system of equations \begin{align*} (ab-1)^2&=c(a^2+b^2)+ab+1,\\ a^2+b^2&=c^2+ab. \end{align*}a) Prove that $c+1$ is a perfect square. b) Find all such triples $(a,b,c)$. Summing these two equalities we get \[ c^2-c(a^2+b^2)+(a^2b^2-2ab-a^2-b^2)=0. \]Note that $\Delta_c=(a+b)^2((a-b)^2+4)$ must be perfect square, so $(a-b)^2+4$ is perfect square. Therefore, $a=b$. Putting this in second equality, we get $2a^2=c^2+a^2 \Rightarrow a=c$, i.e. $a=b=c$. So from first equality we get \[ (a^2-1)^2=2a^3+a^2+1 \Rightarrow a^2(a+1)(a-3)=0 \Rightarrow \boxed{a=b=c=3}. \]Now note that $c+1=4=2^2$ is really perfect square.

A solution with GCD We know that $(ab-1)^{2} = (c+1)(a^{2} +b^{2} +1-c)$ Assume $d=gcd((c+1),( a^{2}+b^{2} +1-c))$ d divides $a^{2} +b^{2} +2$ so d divides $c^{2} + ab + 2 $ and d divides $c+1$ so d divides $ab+3$ We knew d divides $ab-1$ and therefore d divides 4 and it means than d= 1or 2 or 4 If $d=1$ so the product of two coprime number is a perfect square therefore each of them are perfect square and we are done Let $d=2k$. We get that a , b , c are odd . Using the first equation, by mod 8 it gives that 8 divides $ab-1$ . Now look at second equation, it gives that 8 divides $2c+2$ therefore 4 divides $c-3$ . We know that 8 divides $a^{2} -1$ when a is odd. So 4 divides $c+1$ and 4 divides $a^{2} + b^{2} +1-c $ so $d=4$ and we are done