$f(n)=n-1$ for $n\ge3$ and $f(2)$ arbitrary works.

"I was disappointed when I solved number 5, because then I had to work on number 6." --Mark Sellke, MOP 2013

Let $P(a,b,c)$ be the assertion that $f^{abc-a}(abc)+f^{abc-b}(abc)+f^{abc-c}(abc) = a+b+c$ Claim 1: $f^{x^2-x}(x^2) = x$ for all $x>1$. $P(x,x,x)$: $f^{x^3-x}(x^3) = x$ for all $x>1$ Thus $f^{x^9-x^3}(x^9) = x^3$, so $f^{x^9-x}(x^9) = f^{x^3-x}(x^3) = x$ for all $x>1$. $P(x,x^3,x^5)$: $f^{x^9-x}(x^9) +f^{x^9-x^3}(x^9) +f^{x^9-x^5}(x^9) = x+x^3+x^5 \implies f^{x^9-x^5}(x^9)=x^5$ $P(x^5,x^2,x^2)$: $f^{x^9-x^5}(x^9) + 2 f^{x^9-x^2}(x^9)= x^5 + 2x^2\implies f^{x^9-x^2}(x^9)=x^2$ Thus $f^{x^2-x}(x^2) = f^{x^2-x}(f^{x^9-x}(x^9)) = f^{x^9-x}(x^9) = x$. Claim 2: $f^{ab-a}(ab) + f^{ab-b}(ab) = a+b$ for all $a,b>2$ $P(a,b,ab)$: $f^{a^2b^2-a}(a^2b^2) + f^{a^2b^2-b}(a^2b^2) + f^{a^2b^2-ab}(a^2b^2) = a+b+ab$ Now, $f^{ab-a}(ab) = f^{ab-a}(f^{a^2b^2-ab}(a^2b^2)) = f^{a^2b^2-a}(a^2b^2)$, so we get $f^{ab-a}(ab) + f^{ab-b}(ab) = f^{a^2b^2-a}(a^2b^2) + f^{a^2b^2-b}(a^2b^2) = a+b$, as desired. Claim 3: For any $a,b,c>1$, $(f^{abc-a}(abc)-a) + (f^{abc-b}(abc)-b) = (f^{abc-ab}(abc)-ab)$ By Claim 2, $f^{abc-ab}(abc)+f^{abc-c}(abc) = ab+c$, and by the original given, $f^{abc-a}(abc)+f^{abc-b}(abc)+f^{abc-c}(abc) = a+b+c$. Thus $(f^{abc-a}(abc)-a) + (f^{abc-b}(abc)-b) = c - f^{abc-c}(abc) = (f^{abc-ab}(abc)-ab)$ Claim 4: For any $n>1$ and $a_1,a_2,\ldots,a_n>1$, if $P=a_1a_2\cdots a_n$, then \[\sum_{i=1}^n (f^{P-a_i} (P) - a_i) = 0\] We proceed by induction. The above claim is clearly true for $n=2$ and $n=3$. Suppose the claim is true for all $n=k-1$, and consider $a_1,a_2,\ldots,a_k>1$, and let $P = a_1a_2\cdots a_k$. By the inductive hypothesis on the sequence $a_1,a_2,\ldots,a_{k-2},a_{k-1}a_k$, \[\sum_{i=1}^{k-2} (f^{P-a_i} (P) - a_i) + (f^{P-a_{k-1}a_k} (P) - a_{k-1}a_k)= 0\] However, by Claim 3 on the triple $(a_{k-1},a_k,P/a_ka_{k-1})$, we have \[0 = \sum_{i=1}^{k-2} (f^{P-a_i} (P) - a_i) + (f^{P-a_{k-1}a_k} (P) - a_{k-1}a_k)=\sum_{i=1}^k (f^{P-a_i} (P) - a_i)\] which completes the induction Claim 5: $f(x) = x-1$ for all $x>2$. By Claim 4, we have that for all $r\ge1$, \[rf^{x^r(x-1)-x}(x^r(x-1)) + f^{x^r(x-1)-(x-1)}(x^r(x-1)) = rx+(x-1) \] Let $x_r = f^{x^r(x-1)-x}(x^r(x-1))$. Then this equation becomes $rx_r + f(x_r) = rx+(x-1)$. Notice that for all $r>x$, \[rx_r < rx_r + f(x_r) = rx+(x-1) < rx + r < r(x+1)\] Thus for $r>x$, $x_r\le x$. In particular, by the Pigeonhole Principle, there is a particular value $y\le x$ for which there are infinitely many $r$ such that $x_r = y$. Since $ry + f(y) = rx + (x-1)$ for infinitely many values of $r$, it must be the case that $y=x$, by which $rx+f(x) = rx+(x-1)$ implies that $f(x) = x-1$ for all $x>3$, as desired. As manuel153 pointed out, it is easy to check that any $f$ satisfying $f(x) = x-1$ for $x>2$ and $f(1),f(2)$ arbitrary satisfies the original functional equation.

For $n\ge2$, let $a=n,b=n+1,N=a^rb^s, g(x)=f^{N^9-x}(N^9)-x, h(x)=f^{N^3-x}(N^3)-x$. Then if $xyz=N^9, g(x)+g(y)+g(z)=0$, and if $xyz=N^3,h(x)+h(y)+h(z)=0.$ Clearly $g(N^3)=0$, and if $x<N^3$, $f^{N^9-x}(N^9)=f^{N^3-x}(f^{N^9-N^3}(N^9))=f^{N^3-x}(N^3)$, so $g(x)=h(x)$. Thus $g(N)=h(N)=0$, and since $N^9=N^5N^3N=N^5N^2N^2=N^6N^2N$, $g(N^5)=g(N^2)=g(N^6)=0.$ Let $x,y \mid N, x,y \ge 2$. Then $g(x)+g(y)+g(N^3/xy)=h(x)+h(y)+h(N^3/xy)=0$, and $g(N^6)+g(xy)+g(N^3/xy)=0$, so $g(xy)=g(x)+g(y)$. So $0=g(N)=rg(a)+sg(b)$. But $g(a),g(b) \ge -n$, so if $r,s$ are relatively prime and greater than $n$, we must have $g(a)=g(b)=0$, which means that $f(n+1)=n$.

The answer is $f(n) = n-1$ for $n \ge 3$ with $f(1)$ and $f(2)$ arbitrary; check these work. Lemma: We have $f^{t^2-t}(t^2) = t$ for any $t \ge 2$ Proof: We say $1 \le k \le 7$ is good if $f^{t^9-t^k}(t^9) = t^k$. First, we observe that \[ f^{t^9-t^3}(t^9) = t^3 \quad\text{and}\quad f^{t^3-t}(t^3) = t \implies f^{t^9-t}(t^9) = t. \] so $k=1$ and $k=3$ are good. Then taking $(a,b,c) = (t,t^4,t^4)$, $(a,b,c) = (t,t^3,t^5)$, $(a,b,c) = (t^2, t^2, t^5)$ gives that $4$, $5$, $2$ are good. The lemma follows from this. $\blacksquare$ Now, letting $t = abc$ we combine \begin{align*} f^{t-a}(a) + f^{t-b}(b) + f^{t-c}(c) &= a+b+c \\ f^{t^2-ab} (t^2) + f^{t^2-t} (t^2) + f^{t^2-c} (t^2) &= ab + t + c \\ \implies \left[ f^{t-a}(t) - a \right] + \left[ f^{t-b}(t) - b \right] &= \left[ f^{t-ab}(t) - ab \right] \end{align*} by subtracting and applying the lemma repeatedly. Thus if $p > q > \max\{a,b\}$ are primes then upon setting $s = a^p b^q$ and $t = s^2$ we get \[ p \left[ f^{t-a}(t) - a \right] + q \left[ f^{t-b}(t) - b \right] = \left[ f^{t-s}(s) - s \right] = 0. \] As $q \mid f^{t-a}(t) - a$, $p \mid f^{t-b}(t) - b$, we can conclude that $f^{t-a}(t) = a$ and $f^{t-b}(t) = b$ for any $a,b \ge 2$, with $t = a^p b^q$. In particular, if $a = n$ and $b = n+1$ then we deduce $f(n+1) = n$ for all $n \ge 2$, as desired.

v_Enhance wrote: The answer is $f(n) = n-1$ for $n \ge 3$ with $f(1)$ and $f(2)$ arbitrary; check these work. Lemma: We have $f^{t^2-t}(t^2) = t$ for any $t \ge 2$ Proof: We say $1 \le k \le 7$ is good if $f^{t^9-t^k}(t^9) = t^k$. First, we observe that \[ f^{t^9-t^3}(t^9) = t^3 \quad\text{and}\quad f^{t^3-t}(t^3) = t \implies f^{t^9-t}(t^9) = t. \]so $k=1$ and $k=3$ are good. Then taking $(a,b,c) = (t,t^4,t^4)$, $(a,b,c) = (t,t^3,t^5)$, $(a,b,c) = (t^2, t^2, t^5)$ gives that $4$, $5$, $2$ are good. The lemma follows from this. $\blacksquare$ Now, letting $t = abc$ we combine \begin{align*} f^{t-a}(a) + f^{t-b}(b) + f^{t-c}(c) &= a+b+c \\ f^{t^2-ab} (t^2) + f^{t^2-t} (t^2) + f^{t^2-c} (t^2) &= ab + t + c \\ \implies \left[ f^{t-a}(t) - a \right] + \left[ f^{t-b}(t) - b \right] &= \left[ f^{t-ab}(t) - ab \right] \end{align*}by subtracting and applying the lemma repeatedly. Thus if $p > q > \max\{a,b\}$ are primes then upon setting $s = a^p b^q$ and $t = s^2$ we get \[ p \left[ f^{t-a}(t) - a \right] + q \left[ f^{t-b}(t) - b \right] = \left[ f^{t-s}(s) - s \right] = 0. \]As $q \mid f^{t-a}(t) - a$, $p \mid f^{t-b}(t) - b$, we can conclude that $f^{t-a}(t) = a$ and $f^{t-b}(t) = b$ for any $a,b \ge 2$, with $t = a^p b^q$. In particular, if $a = n$ and $b = n+1$ then we deduce $f(n+1) = n$ for all $n \ge 2$, as desired. How do you deduce $f^{t-a}(t) = a$? When $p,q$ vary, $t$ is changing.

It's not a variation argument. Since $f^{t-a}(t) - a \ge -a > -q$ but also is divisible by $q$, we have $f^{t-a}(t) - a \ge 0$ ad similarly $f^{t-b}(t) - b \ge 0$. But the weighted sum is zero, so both must equal zero.

v_Enhance wrote: It's not a variation argument. Since $f^{t-a}(t) - a \ge -a > -q$ but also is divisible by $q$, we have $f^{t-a}(t) - a \ge 0$ ad similarly $f^{t-b}(t) - b \ge 0$. But the weighted sum is zero, so both must equal zero. Nice! I've got it.

This was a problem our teacher had just talked about.......

v_Enhance wrote: Let $\mathbb N$ be the set of positive integers. Find all functions $f: \mathbb N \to \mathbb N$ that satisfy the equation \[ f^{abc-a}(abc) + f^{abc-b}(abc) + f^{abc-c}(abc) = a + b + c \]for all $a,b,c \ge 2$. (Here $f^1(n) = f(n)$ and $f^k(n) = f(f^{k-1}(n))$ for every integer $k$ greater than $1$.) Solution. The solution is $f(n)=n-1$ for all $n\ge 3,$ with $f(1)$ and $f(2)$ arbitrary natural numbers, it is easy to check that this works. Let $P(a,b,c)$ be the given assertion. We prove some claims: Claim 1. Let $x_1,x_2,\ldots,x_{n-2}$ be integers such that \[x_i+x_j+x_k=0\text{ for all }i+j+k=n,~i\ge j\ge k\ge 1\]Then it follows that $x_k=\frac{A(n-3k)}{n-3}$ for some constant $A\in\mathbb{Z}.$

Claim 2. $f^{n^a-n^b}(n^a)=n^b$ for all $n\ge 2, a\ge b.$

Claim 3. $f^{n^A(n+1)^B-n^a(n+1)^b}(n^A(n+1)^B)=n^a(n+1)^b$ where $n\ge 2,~a+b\ge 1, A>a, B>b, \tfrac{B}{A}\in\mathbb N_{\ge 2}$ and $A,B$ are large.

Coming back to the original problem, let $T=n^A(n+1)^{2A}$ where $A$ is large and $n\ge 2.$ By claim 3 we have \begin{align*} f^{T-n}(T) &= n,\\ f^{T-(n+1)}(T) &=n+1. \end{align*}Therefore, \[f(n+1)=f(f^{T-(n+1)}(T))=f^{T-n}(T) = n.\]And we are done. $\blacksquare$

Very enjoyable problem! The answers are $f(n)=n-1$ for all $n\geq 3$ where $f(1)$ and $f(2)$ arbitrary. It's straightforward to check that these solutions work, hence omitted. We divide the solution into 3 steps. Step 1: Random wander The central result of this section is the following. Claim: $f^{n^2-n}(n^2) = n$ for every $n\geq 2$. Proof: Let $P(a,b,c)$ denote the above assertion. We do this by a series of substitutions. $P(n,n,n)\implies f^{n^3-n}(n^3)=n$. Replacing $n$ with $n^3$ gives $f^{n^9-n}(n^9)=n$. Thus $P(n,n^4,n^4)\implies f^{n^9-n^4}(n^9) = n^4$. Hence combining with above gives $f^{n^4-n}(n^4)=n$. Finally, $P(n,n,n^2)\implies f^{n^4-n^2}(n^4)=n^2$. Combining with the above point gives the claim. $\blacksquare$ Step 2: The equation generalizes itself. Claim: (The $n=2$ case) $f^{ab-a}(ab)+f^{ab-b}(ab) = a+b$ for any $a,b\geq 2$. Proof: From $P(a,b,ab)$, we get $$f^{(ab)^2-ab}((ab)^2) + f^{(ab)^2-a}((ab)^2)+f^{(ab)^2-b}{(ab)^2} = ab+a+b$$Using the Claim from Step 1, this reduces to $ab + f^{ab-a}(ab) + f^{ab-b}(ab) = ab+a+b$ so we are done. $\blacksquare$ Claim: For any $x_1,x_2,\hdots,x_n\geq 2$ with product $P$, we have $$f^{P-x_1}(P) + f^{P-x_2}(P) + \hdots + f^{P-x_n}(P) = x_1+x_2+\hdots+x_n$$Proof: Fix a positive integer $N$, define $g(a) = f^{N-a}(N)-a$. We note that if $N = abc$, then \begin{align*} \text{Claim}&\implies g(ab) + g(c) = 0\\ P(a,b,c) &\implies g(a)+g(b)+g(c) = 0 \end{align*}Thus $g(ab)=g(a)+g(b)$ whenever $ab$ is a proper divisor of $N$. Thus we let $N=P$, then $$g(x_1)+g(x_2)+\hdots+g(x_{n-1})+g(x_n) = g(x_1x_2\hdots x_{n-1}) + g(x_n) = 0$$which implies the claim. $\blacksquare$ Step 3: Finale Fix two positive integers $a,b\geq 2$ and pick large primes $p,q$. From step 2 if $N=a^pb^q$, we get $$pf^{N-a}(N) + qf^{N-b}(N) = ap+bq$$Thus $f^{N-a}(N) < a+b$ Thus $|f^{N-a}(N) - a| < \max\{a,b\}$. However, since $q\mid f^{N-a}(N) - a$, we get that $f^{N-a}(N) = a$ and thus $f^{N-b}(N)=b$. In particular, picking $b=a+1$ gives $f(a+1)=a$ for any $a\geq 2$. This implies that all solutions are of the form described at the beginning.

I think this problem is really nice. Our goal is to prove that $f^{ab-a}(ab) = a$ holds for all $a,b\ge 2$, from which it is easy to deduce the solution set. First, because of the multiplicative structure, we deal with integer powers first. Plug in $(a,b,c) = (n,n,n)$ to get \[f^{n^3-n}(n^3) = n.\] Claim. For any $n\ge 2$, $f^{n^k-n^l}(n^k) = n^l$ for any integers $k>l\ge 1$. Proof. We say $P(k,l)$ is true if the above equation holds for $k>l\ge 1$. By definition, the following are true: For $k>l>m\ge 1$, $P(k,l)$ and $P(l,m)$ implies $P(k,m)$; For $k>l>m\ge 1$, $P(k,l)$ and $P(k,m)$ implies $P(l,m)$; For $k = a+b+c$ where $a,b,c\ge 1$, $P(k,a)$ and $P(k,b)$ implies $P(k,c)$. By item 2, it suffices to show that $P(k,l)\ \forall 1\le l\le k-2$ for arbitrarily large $k$. Take $k = 3^i$ to be a power of 3, and induct on $i$. We first prove the base case where $k = 9$. We have $P(3,1)$ and $P(9,3)$, so $P(9,1)$ is also true. By item 3, we have $P(9,4), P(9,2), P(9,5), P(9,6)$, and $P(9,7)$ as desired. For the induction step, suppose $P(3^i,l)$ is true for all $1\le l\le k-2$. To start with, we have $P(3^{i+1},3^i)$, so by item (1) we deduce $P(3^{i+1},l)$ for \[l \in L := \{1,2,3,\dots, 3^i-2,3^i\}.\]But $L+L = \{2,3,\dots,2\cdot 3^i - 2, 2\cdot 3^i\}$, so by item (3) we deduce $P(3^{i+1},l)$ is also true for all $l\in L' := \{3^i, 3^i + 2,\dots,3^{i+1} - 2\}$. But $L\cup L' = \{1,2,\dots,3^{i+1}-2\}$, completing the induction step. $\square$ An immediate consequence of the above claim is that whenever we see something like $f^{M^N - a}(M^N)$ for $a\mid M^N$, it is equal to $f^{M^n - a}(M^n)$ for any $n$ as long as $M^n > a$. The next step is to consider a slight generalization of integer powers: expressions like $a^kb^l$ for two distinct integers $a,b\ge 2$. Clearly, \[a + b + ab = f^{a^2b^2-ab}(a^2b^2) + f^{a^2b^2-a}(a^2b^2) + f^{a^2b^2-b}(a^2b^2) = ab + f^{ab-a}(ab) + f^{ab-b}(ab)\]so \[f^{ab-a}(ab) + f^{ab-b}(ab) = a + b.\]We use the idea of so-called "error propagation". Suppose $f^{ab-a}(ab) = a - t$ and $f^{ab-b}(ab) = b + t$ for some suitable integer $t$. Claim. For any integers $k,l\ge 1$, and $N > k,l$, we have \[f^{a^Nb^N - a^kb^l}(a^Nb^N) = a^kb^l - (k-l)t.\]Proof. The idea is still induction on the exponents. We first prove a weaker equation, that is \[f^{a^kb^k - a^k}(a^kb^k) = a^k - kt.\]Induct on $k$, the base case is trivial. For the induction step, plug in, say, $(a,b,c) = (a^k, b, b^{k-1})$ and use inductive hypothesis to get the above equation. For the equation in the claim, WLOG $k>l$, then plug in, say, $(a,b,c) = (a^kb^l, b^{k-l}, ab)$. $\square$ The upshot is that many integers $n$ do not have a unique representation as $n = a^kb^l$. For example, if $k>l$, then $a^kb^l = a^{k-l}(ab)^l$. This means that \[(k-l)t_{a,b} = (k-2l)t_{a,ab}\]where $t_{a,b}$ represents the value of $t$ depending on $a,b$. Taking $k,l$ sufficiently large with $\gcd(k-l,l) = 1$, we see that if $t_{a,b}\neq 0$ then $t_{a,b}$ is not bounded above, which is clearly impossible. Thus $t_{a,b} = 0$ for any $a,b\ge 2$. This means that $f^{ab-a}(ab) = a$. At this point, the original functional equation can no longer provide any information, so we must be pretty close to getting the solution set. Indeed, taking $b=a+1$, then we have $a = f^{a(a+1)-a}(a(a+1)) = f(f^{a(a+1)-a-1}(a(a+1))) = f(a+1)$, so $f(a+1) = a$ for any $a\ge 2$. The value of $f(1)$ and $f(2)$ are arbitrary, and we can check that this indeed works. $\square$

The answer is $f(1), f(2)$ arbitary and $f(n)=n-1\forall n\ge 3$. They clearly work. Let $P(a,b,c)$ denote the assertion in the problem. $P(x,x,x)$ gives $f^{x^3-x}(x^3)=x$ $P(x^3,x^3,x^3)$ gives $f^{x^9-x^3}(x^9)=x^3$, so $f^{x^9-x}(x^9)=x$ $P(x,x^3,x^5)$ gives $f^{x^9-x^5}(x^9)=x^5$, so $f^{x^5-x}(x^5)=f^{x^5-x}(f^{x^9-x^5}(x^9))=f^{x^9-x}(x^9)=x$. I claim $f^{x^k-x}(x^k)=x$ for all $k$. Let $S: \{ k\ge 2: f^{x^k-x}(x^k)=x \}$ Suppose $a,b,a+b+c\in S$, then $P(x^a,x^b,x^c)$ yields $c\in S$. Moreover, if $k\in S$, any multiple of $k$ is in S. Suppose all odd numbers in $\{1,\cdots,3^k\}$ are in $S$, then all multiples of $3$ in $\{1,\cdots,3^{k+1}\}$ are also in $S$, so it's clear that $S$ contains all the odd numbers. Finally, $P(x,x^k,x^k)$ yields $S$ also contains even numbers. With this in mind, $P(b,c,bc)$ yields $bc+b+c=f^{b^2c^2-bc}(b^2c^2) + f^{b^2c^2-b}(b^2c^2) + f^{b^2c^2-c}(b^2c^2) = bc + f^{bc-b}(bc)+f^{bc-c}(bc)$, so $f^{bc-b}(bc)+f^{bc-c}(bc)=b+c$. Let $Q(b,c)$ denote the assertion to the left. Note $Q(ab,c)-P(a,b,c)$ gives $f^{abc-ab}(abc) - f^{abc-a}(abc) - f^{abc-b}(abc) = ab-a-b$, denote $R(a,b,c)$. Splitting $b=de$, we can see $f^{acde-ade}(acde)-f^{acde-a}(acde)-f^{acde-de}(acde)=ade-a-de$. Furthermore, $f^{acde-de}(acde)-f^{acde-d}(acde)-f^{acde-e}(acde)=de-d-e$ by $R(d,e,ac)$. We can use this to induct to see if $P=\prod\limits_{i=1}^n a_i$, then $f^{cP-P}(cP) - \sum\limits_{i=1}^n f^{cP-a_i}(cP) = P-\sum\limits_{i=1}^n a_i$. In particular, this argument works even when $c=1$, so we can see $\sum\limits_{i=1}^n f^{P-a_i}(P)=\sum\limits_{i=1}^n a_i$ for all $2\le a_1,\cdots,a_n$. Furthermore, if $c>1$, we can see that $\sum\limits_{i=1}^n a_i + c = \sum\limits_{i=1}^n f^{cP-a_i}(cP) + f^{cP-c}(cP).$ This implies $f^{cP-c}(cP)=c$ by subtraction. This should hold for all $P$ composite. Lastly, $f^{cdP-cd}(cdP)=cd$, so $c=f^{cdP-c}(cdP)=f^{cd-c}(f^{cdP-cd}(cdP))=f^{cd-c}(cd)$, so this holds for all $P\ge 2$. Picking $c(c+1)\mid P$ suggests $f^{P-c}(P)=c, f^{P-c-1}(P)=c+1$, so $f(c+1)=c$, as desired.

MarkBcc168 wrote: Very enjoyable problem! no Let $P(a,b,c)$ denote the assertion. $\textbf{Claim: }$ $f^{a^2-a}(a^2)=a$ $\emph{Proof: }$ $P(a,a,a)$ and $P(a^3,a^3,a^3)$ give $f^{a^3-a}(a^3)=a$ and $f^{a^9-a^3}(a^9)=a^3,$ so $$f^{a^9-a}(a^9)=f^{a^3-a}(f^{a^9-a^3}(a^9))=f^{a^3-a}(a^3)=a.$$Then, $P(a,a,a^7)$ yields $$2f^{a^9-a}(a^9)+f^{a^9-a}(a^9)=2a+a^7\implies f^{a^9-a^7}(a^9)=a^7$$Similarly, we can show that $f^{a^9-a^5}(a^9)=a^5,$ so from $P(a^2,a^2,a^5),$ we have $$2f^{a^9-a^2}(a^9)+f^{a^9-a^5}(a^9)=2a^2+a^5\implies f^{a^9-a^2}(a^9)=a^2$$$$\implies f^{a^2-a}(a^2)=f^{a^9-a}(a^9)=a,$$as desired. $\blacksquare$ Note that $$f^{b^{2}c^{2}-b}(b^{2}c^{2})=f^{bc-b}(f^{b^{2}c^{2}-bc}(b^{2}c^{2}))=f^{bc-b}(bc),$$and similarly, $f^{b^{2}c^{2}-c}=f^{bc-c}(bc).$ Now, $P(bc,b,c)$ implies that $$f^{bc-b}(bc)+f^{bc-c}(bc)=b+c.\hspace{25pt}(1)$$Plugging in $(a,bc)$ and comparing with $P(a,b,c),$ we find that $$f^{abc-bc}(abc)-bc=(f^{abc-b}(b)-b)+(f^{abc-c}(c)-c)\hspace{25pt}(2).$$Let $d\ge 2$ be a positive integer, and define a function $g$ by $$g(x)=f^{d^{m}(d+1)^{n}-x}(d^{m}(d+1)^{n})-x,$$where $m,n> d+1$ are coprime. By (1), we have $$g(d^m)+g((d+1)^n)=0.$$Moreover, by (2), we know $g(x)+g(y)=g(xy)$ for all $x,y,$ so $$mg(d)+ng(d+1)=0.$$Hence, $m\mid g(d+1).$ But $g(d+1)<d+1,$ so we must have $g(d+1)=0.$ We can similarly show that $g(d)=0,$ so $f(d+1)=d.$ This yields the solution $f(n)=n-1$ for $n>2$ and $f(n)$ is any positive integer for $n=1,2,$ which can be easily checked to work.

We will show that the only solution is $f(x)=x-1$ for all $x>2$ with $f(1)$ and $f(2)$ some constant. Denote the assertion by $P(a,b,c).$ Claim: $f^{x^2-x}(x^2)=x.$ Proof. $P(x,x,x)$ gives $f^{x^3-x}(x^3)=x.$ Thus $f^{x^9-x^3}(x^9)=x^3.$ So $f^{x^9-x}(x^9)=x.$ We have $f^{x^9-x^4}(x^9)=x^4$ from $P(x^4,x^4,x).$ So $f^{x^4-x}(x^4)=x.$ Note that $f^{x^4-x^2}(x^4)=x^2$ from $P(x^2,x,x).$ Now just compare. $\blacksquare$ Comparing $P(abc,ab,c)$ with $P(a,b,c)$ we get $f^{abc-ab}(abc)-abc=f^{abc-a} (abc)-a+ f^{abc-b}(abc)-b.$ Pick large primes $p,q.$ Fixing $a,b$ and setting $abc=(a^pb^q)^2=x^2$ in the relation gives $0=f^{x^2-x}(x)-x=p(f^{x^2-a}(x^2)-a+q(f^{x^2-b}(x^2)-b).$ Since $p\mid f^{x^2-b}(x^2)-b$ we must have $f^{x^2-a}(x^2)=a.$ Setting $a=b+1$ we get our solutions.