5 students solved it during the contest

The oficial solution used the fact that the rolling polygon is the reflection of the fixed polygon in each side. Thus the point marked must be the reflection of one vertex of the fixed polygon in each side of the fixed polygon, and we had to calculate the area of the broken line formed by these reflections. Now the rest is not so difficult. All one has to do is divide this region inside the broken line into a number of triangles such that these triangles can be combined to give the required answer.

This one looks like a nice and hard problem problem indeed. I wonder if the limiting case when $n \to \infty$ can be used to find the area of a cardioid?

Mindstormer wrote: This one looks like a nice and hard problem problem indeed. The problem isn't that hard, it's nice though. I'll post a solution soon. Mindstormer wrote: I wonder if the limiting case when $n \to \infty$ can be used to find the area of a cardioid? That's the first thing I thought of when I saw the problem And yes, the limiting case does give the area of a cardoid. Just have to changing the regular $n$-gon of unit side to unit radius and need to scale the final area appropriately.

Label the vertices of the regular $n$-gon as $A_0A_1\cdots A_{n-1}$. WLOG let $A_0$ be the initial vertex of the rolling polygon. When the rolling polygon has reached the side $\overline{A_jA_{j+1}}$, the moving vertex is placed on the $j$th vertex of this rolling $n$-gon from $A_j$ ( anti-clockwise). Call this vertex $B_j$. Since, at this moment, the two polygons are reflections about $\overline{A_jA_{j+1}}$ the point $B_j$ is the reflection of $A_0$ over $\overline{A_jA_{j+1}}$. Hence we can write \begin{align*} [\kappa] &=\sum_{j=0}^{n-1}\left( [A_jA_0A_{j+1}]+[A_jB_jA_{j+1}]\right)+\sum_{j=0}^{n-1}[B_jA_{j+1}B_{j+1}] \end{align*} Notice that $[A_jA_0A_{j+1}]=[A_jB_jA_{j+1}]$ and $B_jA_{j+1}=B_{j+1}A_{j+1}=A_0A_{j+1}$ as they are reflections over $A_jA_{j+1}$ and $\angle{B_jA_{j+1}B_{j+1}}=\dfrac{4\pi}{n}.$ Thus we have \begin{align*} [\kappa] &=2A+\dfrac{1}{2}\sum_{j=1}^{n-1}\overline{A_0A_j}^2\sin\dfrac{4\pi}{n}. \end{align*}By similarity we have $B=4A\sin^2\dfrac{\pi}{n}$ and using $A=\dfrac{n}{4}\cot\dfrac{\pi}{n}$, it suffices to prove that \begin{align*} \dfrac{1}{2}\sum_{j=1}^{n-1}\overline{A_0A_j}^2\sin\dfrac{4\pi}{n}=4A-2B &=\left(4-8\sin^2{\dfrac{\pi}{n}}\right)A \\ \sum_{j=1}^{n-1}\overline{A_0A_j}^2\cos\dfrac{2\pi}{n}\sin\dfrac{2\pi}{n} &=4\cos{\dfrac{2\pi}{n}}A \\ \sum_{j=1}^{n-1}\overline{A_0A_j}^2\sin{\dfrac{2\pi}{n}} &=4A. \end{align*} Assign the complex number $A_k=r\omega^k$ where $\omega$ is a $n$-th root of unity and $r=\dfrac{1}{2\sin{\dfrac{\pi}{n}}}$. Then we have \begin{align*} \sum_{j=1}^{n-1}\overline{A_0A_j}^2 &=\sum_{j=1}^{n-1}|r-r\omega^j|^2 \\ &=r^2\sum_{j=1}^{n-1}\left(1-\omega^j\right)\left(1-\overline{\omega}^j\right) \\ &=r^2\sum_{j=1}^{n-1}\left(1-\omega^{j}-\overline{\omega}^j+1\right)\\ &=2nr^2, \end{align*}since $\sum \omega^j=\sum \overline{\omega}^j=0$. Comparing the terms, it is easy to see that $nr^2\sin{\dfrac{2\pi}{n}}=2A$, giving us our desired result.