Denote by $A(M)$ the number of solutions for some matrix $M$ (with some entries already prescribed). Obviously, $1$ cannot lie in the center of the square, since it would have four neighbours, but $1$ can only neighbour $2,3,4$. Hence, the number of solutions is \[4A\begin{pmatrix}1&*&*\\ *&*&*\\ *&*&*\end{pmatrix}+4A\begin{pmatrix}*&1&*\\ *&*&*\\ *&*&*\end{pmatrix}.\]Note that in the Taxicab metric, $1$ and $9$ have to have distance at least $3$. Similarly, $1,8$ and $2,9$ have distance at least $3$. Also, any two numbers with difference $6$ (say, $1$ and $7$) cannot share a corner, since then there are two paths of length $2$ connecting them, where we have to add $3$ each time and thus reuse a number. Since $8,9$ have distance $\geq 3$ to $1$, it follows that \[4A\begin{pmatrix}*&1&*\\ *&*&*\\ *&*&*\end{pmatrix}=8A\begin{pmatrix}*&1&*\\ *&*&*\\ 8&*&9\end{pmatrix}.\]By above discussion, $7$ cannot share a corner with $1$, so it has to be between the $8$ and $9$ with a $4$ seperating it from $1$. Then $9$ has another neighbour, which must be $6$, so \[8A\begin{pmatrix}*&1&*\\ *&*&*\\ 8&*&9\end{pmatrix}=8A\begin{pmatrix}*&1&*\\ *&4&6\\ 8&7&9\end{pmatrix}.\]Now there is only one possibility to place $2$, after which only one possibility for $3$ remains: \[8A\begin{pmatrix}*&1&*\\ *&4&6\\ 8&7&9\end{pmatrix}=8A\begin{pmatrix}2&1&3\\ 5&4&6\\ 8&7&9\end{pmatrix}=8.\] We now consider the case that $1$ is in the corner. Since $9$ has distance $\geq 3$, it follows that \[4A\begin{pmatrix}1&*&*\\ *&*&*\\ *&*&*\end{pmatrix}=4A\begin{pmatrix}1&*&*\\ *&*&*\\ *&*&9\end{pmatrix}+8A\begin{pmatrix}1&*&*\\ *&*&*\\ *&9&*\end{pmatrix}.\]Now in the latter case, $9$ has three neighbours, which must be $6,7,8$. Since $1$ and $8$ have distance $\geq 3$ and $1$ and $7$ don't share a corner, this forces \[8A\begin{pmatrix}1&*&*\\ 4&6&*\\ 7&9&8\end{pmatrix}.\]Again, the placement $4$ follows because $1$ and $7$ have distance $2$. Now the placement of $5$ is forced next to $8$, after which $2$ has only one possibility, so \[8A\begin{pmatrix}1&3&2\\ 4&6&5\\ 7&9&8\end{pmatrix}=8.\] Now consider the case where both $1$ and $9$ are in corners. Since $1$ and $8$ have distance $\geq 3$ and $2,8$ cannot share a corner, we have (the $5$ in the center is again forced) \[4A\begin{pmatrix}1&*&*\\ *&*&*\\ *&*&9\end{pmatrix}=8A\begin{pmatrix}1&2&*\\ *&5&*\\ *&8&9\end{pmatrix}=8A\begin{pmatrix}1&2&*\\ 3&5&*\\ *&8&9\end{pmatrix}+8A\begin{pmatrix}1&2&*\\ 4&5&*\\ *&8&9\end{pmatrix}.\]For the first matrix, there is only one choice for $4$ and after that only one choice for $7$. For the second matrix, there is only one choice for $3$ and after that one choice for $7$. All in all, we have \[4A\begin{pmatrix}1&*&*\\ *&*&*\\ *&*&9\end{pmatrix}=8A\begin{pmatrix}1&2&4\\ 3&5&7\\ 6&8&9\end{pmatrix}+8A\begin{pmatrix}1&2&3\\ 4&5&6\\ 7&8&9\end{pmatrix}.\] Finally, we count $32$ total solutions.