Let $ST,PS$ meet $\Omega$ at $M,K$, Hence $YM\parallel XS$, which means that $YM\perp PK$ $\Rightarrow$ $M$ is the midpoint of arc$PK$. Since $\angle POT=2(180-\angle PST)=\angle SXT=\angle MYT$, hence $\triangle POT\sim \triangle SXT\sim \triangle MYT$. By their spiral similarity transformation, we obtain $PTSM\sim OTXY$. Since $PM^2=MS*MT$($\angle MPK=\angle MKP=\angle PTM$). Hence from $PTSM\rightarrow OTXY$(just angle chasing) we have $OY^2=YX*YT=$constant. Since $Y$ is fixed and the value of $OY$ is constant, hence the locus of $O$ is a circle centered at $Y$ with radius $\sqrt {YX*YT}$. Note that $\angle PMT$ varies from $0$ to $180$ depending on which direction $P$ approaches $T$, this means that $\angle OYT$ also satisfies this, hence $O$ only travels on the half circle with diameter on $TY$.

This problem is very similar to problem 5 of Turkey Team Selection Test 2002 at this link http://www.artofproblemsolving.com/Forum/viewtopic.php?p=3006277&sid=52a1bd7941586703ea04862b89c35c9e#p3006277 where it is asked to prove that $ PM^2 = MS*MT $, and with triangles $MPS$ and $YOX$ similar, we have $ YO^2 = YX*YT $.

Let $P'$ be the second intersection of $PS$ with $\Omega$. Let $M$ be the second intersection of $TS$ with $\Omega$. There is a homothety centered at $T$ that sends $S$ to $M$ and $X$ to $Y$, so $\triangle TXS \sim \triangle TYM$. Also, $M$ is the midpoint of arc $PP'$. So angle chasing yields that $\triangle TP'M \sim \triangle TSP$. Therefore, $\triangle TXS \sim \triangle TYM \sim \triangle TOP$. Therefore there is a spiral similarity centered at $T$ mapping $\triangle OXY$ to $\triangle PSM$. Since $\triangle PSM \sim \triangle TPM$, we have that $\triangle OXY\sim \triangle TOY$. Therefore $OY^2=YX\cdot YT$ which is fixed. It is easy to verify that the locus is then a circle centered at $Y$ with radius $\sqrt{YX\cdot YT}$. $\Box$

[asy][asy] unitsize(100); pair T = (0, 0), X = (0, 0.6), Y = (0, 1); pair O = Y + dir(230) * sqrt(abs(Y-X)*abs(Y-T)); pair S = reflect(X, O) * T; pair P = reflect(Y, O) * T; pair H = extension(Y, O, T, P); draw(P--O--T^^S--X--T^^X--Y, gray(0.6)); draw(circle(X, abs(X-T))); draw(circle(Y, abs(Y-T))); draw(H--Y^^O--X); draw(circumcircle(O, X, T), dotted); draw(P--S--T--cycle); dot(T^^X^^Y^^O^^S^^P^^H); label("$T$", T, dir(270)); label("$X$", X, dir(50)); label("$Y$", Y, dir(60)); label("$S$", S, dir(S-X)); label("$P$", P, dir(P-Y)); label("$O$", O, dir(110)); label("$H$", H, dir(H-Y)); [/asy][/asy] The answer is the circle with center $Y$ and radius $\sqrt{YX \cdot YT}$, minus the two points on line $XY$. First we show $O$ must lie on this circle. Let $H = \overline{YO} \cap \overline{TP}$. Then \[\measuredangle HOT = \measuredangle PST = \measuredangle OXT,\]so $\overline{YO}$ is tangent to $(OXT)$, giving $YO^2 = YX \cdot YT$. For the other direction, suppose that $YO^2 = YX \cdot YT$ but $O$ is not on line $XY$. Then let the circle with center $O$ and radius $OT$ intersect $\Gamma$ and $\omega$ at $P \neq T$ and $S \neq T$. Then reversing the angle chase above gives $\overline{PS}$ tangent to $\omega$, as desired. If $O$ is on line $XY$, then we get $P = S = T$ contradiction.

v_Enhance wrote: Circle $\omega$, centered at $X$, is internally tangent to circle $\Omega$, centered at $Y$, at $T$. Let $P$ and $S$ be variable points on $\Omega$ and $\omega$, respectively, such that line $PS$ is tangent to $\omega$ (at $S$). Determine the locus of $O$ -- the circumcenter of triangle $PST$. We will prove that $O$ lies on a circle centered at $Y$ with radius $\sqrt{YX \cdot YT}$. The other direction is just reversing the angle-chase. Extend $\overline{PS}$ to meet $\Omega$ again at $Q \ne P$. Then $\angle OTP=90^{\circ}-\angle TSQ$ and $\angle YTP=-90^{\circ}-\angle TQP$ hence $\angle YTO=\angle QTS$. Moreover $\overline{XO} \perp \overline{TS}$ and $\overline{YO} \perp \overline{TP}$ so $\angle XOY=\angle PTS$. Now since $\overline{TS}$ bisects angle $PTQ$; we conclude that $\angle YOX=\angle YTO$ as desired. $\blacksquare$

Solution from Twitch Solves ISL: The answer is a circle centered at $Y$ with radius $\sqrt{YX \cdot YT}$, minus the two points on line $XY$ itself. We let $PS$ meet $\Omega$ again at $P'$, and let $O'$ be the circumcenter of $\triangle TPS'$. Note that $O'$, $X$, $O$ are collinear on the perpendicular bisector of line $\overline{TS}$ Finally, we let $M$ denote the arc midpoint of $PP'$ which lies on line $TS$ (by homothety). [asy][asy] import graph; size(9cm); pen qqffff = rgb(0.,1.,1.); pen qqwuqq = rgb(0.,0.39215,0.); pen ffdxqq = rgb(1.,0.84313,0.); pen yqqqqq = rgb(0.50196,0.,0.); pair X = (-2.,0.), Y = (2.,0.), T = (-9.50185,0.), P = (7.49229,10.10580), O = (5.46686,-5.82994), M = (-0.78960,11.15843); pair S = (-3.81946,7.27786); pair Pp = (-7.60181,6.33228); pair Op = (-4.49671,1.94937); fill(T--S--P--cycle, palegreen); fill(T--Op--Y--cycle, palegreen); fill(T--S--Pp--cycle, palecyan); fill(T--O--Y--cycle, palecyan); draw(circle(X, 7.50185), linewidth(0.6) + red); draw(circle(Y, 11.50185), linewidth(0.6) + red); draw(circle(Y, 6.78287), linewidth(0.6) + qqwuqq); draw(T--Op--Y--O--T--Y, grey); draw(T--Pp--M--P--T--M, black); draw(Pp--P, black); draw(Op--O, grey); dot("$X$", X, dir((8.005, 21.422))); dot("$Y$", Y, dir((8.687, 21.422))); dot("$T$", T, dir(T-Y)); dot("$S$", S, dir(135)); dot("$P$", P, dir(P-Y)); dot("$O$", O, dir(-45)); dot("$P'$", Pp, dir(Pp-Y)); dot("$O'$", Op, dir(40)); dot("$M$", M, dir((8.912, 22.013))); [/asy][/asy] By three applications of Salmon theorem, we have the following spiral similarities all centered at $T$: \begin{align*} \triangle TSP &\overset{+}{\sim} \triangle TO'Y \\ \triangle TP'S &\overset{+}{\sim} \triangle TYO \\ \triangle TP'P &\overset{+}{\sim} \triangle TO'O. \end{align*}However, the shooting lemma also gives us two similarities: \begin{align*} \triangle TP'M &\overset{+}{\sim} \triangle TSP \\ \triangle TMP &\overset{+}{\sim} \triangle TP'S. \end{align*}Putting everything together, we find that \[ TP'MP \overset{+}{\sim} TO'YO. \]Then by shooting lemma, $YO' = YX \cdot YT$, so $O$ indeed lies on the claimed circle. As the line $\overline{O'O}$ may be any line through $X$ other than line $XY$ (one takes $S$ to be the reflection of $T$ across this line) one concludes the only two non-achievable points are the diametrically opposite ones on line $XY$ of this circle (because this leads to the only degenerate situation where $S=T$).

The answer is a circle centered at $A$ with radius $\sqrt{YX \cdot YT}$. It suffices to show that the length $YO$, or the ratio $\frac{YO}{YT}$, is constant. Denote by $R$ the intersection of $\overline{SP}$ with $\Omega$, while denote $M = \overline{ST} \cap \Omega$, which is the midpoint of arc $\widehat{RP}$ by the Shooting Lemma. Claim. A spiral similarity at $T$ sends $\overline{RS} \to \overline{YO}$. We will show that $\triangle TSO \sim \triangle TRY$, which suffices. Notice that both triangles are isosceles by definition; then $$\angle SOT = 2\angle RPT = \angle RYT,$$implying the result. $\blacksquare$ Henceforth, $$\frac{YO}{YT} = \frac{RS}{RT} = \frac{MS}{MP} = \sqrt{\frac{MS}{MT}}$$by the Shooting Lemma. But $\overline{SX}$ and $\overline{MY}$ are parallel, because they are both perpendicular to $\overline{RS}$ (by definition of tangency and arc midpoints, respectively.) This implies that $$\frac{MS}{MT} = \frac{YX}{YT}$$is constant, from where the desired result follows.

We first fix $S$ and let the tangent from $X$ with respect to $\omega$ hit $\Omega$ at $P_1$ and $P_2$. Let $O_1$ and $O_2$ be the circumcenters of $(TSP_1)$ and $(TSP_2)$. Notice that $\measuredangle{TO_2P_2} = 2\measuredangle{TSP_2} = 2\measuredangle{TSP_1} = \measuredangle{TO_1P_1}$ and the fact that $TO_1P_1$ and $TO_2P_2$ are both isosceles gives $T$ to be the Miquel Point of $O_1O_2P_2P_1$. Additionally, $$\measuredangle{TO_2P_2} = 2\measuredangle{TSP_2} = \measuredangle{TXS}$$so $T$ is the Miquel Point of $O_1XSP_1$ and $XO_2P_2S$ as well. Finally, we also have $TYP_1$ and $TO_2S$ both being isosceles and $\measuredangle{TYP_1} = 2\measuredangle{TP_2S} = \measuredangle{TO_2S}$ so $T$ is the Miquel Point of $YO_2SP_1$ and $YO_1SP_2$. Thus, we have $$\measuredangle{O_1TY} = \measuredangle{O_1TP_1} + \measuredangle{P_1TY} = \measuredangle{XTS}+\measuredangle{STO_1} = \measuredangle{YTO_2}$$and $$2\measuredangle{TYO_1} = 2\measuredangle{TP_2S} = \measuredangle{TO_2S} = 2\measuredangle{TO_2O_1}$$so $T, O_1, O_2, Y$ are conyclic with $YO_1 = YO_2$. Finally, notice that $$\measuredangle{XO_2Y} = \measuredangle{O_1O_2Y} = \measuredangle{O_1TY} = \measuredangle{YTO_2}$$so $O_2YX \sim TYO_2 \implies YO_2^2 = YX\cdot YT$. Because $YX$ and $YT$ are fixed, $O_1$ and $O_2$ lie on a fixed circle centered at $Y$ with radius $\sqrt{YX\cdot YT}$. $\blacksquare$

Let $Q=\Omega\cap\overline{PS}$; we claim $Q$ lies on $\gamma,$ the circle with center $Y$ and radius $\sqrt{YX\cdot YT}.$ Let $Z$ be the center of $(PST)$ and note $$\angle YZT=\tfrac{1}{2}\angle PZT=\angle QST=\tfrac{1}{2}\angle SXT=180-\angle TXZ$$as $\overline{XZ}$ is the perpendicular bisector of $\overline{ST}.$ Hence, $\triangle XYZ\sim\triangle ZYT$ and $YZ^2=YX\cdot YT.$ Suppose $Z'$ lies on $\gamma$ on the opposite side of $\overline{XY}$ as $S.$ Let the circle with radius $\overline{ZT}$ and center $Z$ intersect $\omega$ and $\Omega$ at $S'$ and $P'.$ Then, since $YZ^2=YX\cdot YT,$ we know $\triangle XYZ'\sim\triangle Z'YT.$ Thus, $$180-\angle TSP=\tfrac{1}{2}\angle P'Z'T=\angle YZ'T=\angle Z'XY=\tfrac{1}{2}\angle S'XT$$and $S=S'.$ $\square$

Note that $XO\perp ST$ and $YO\perp PT$ since $XS=XT$ and $YP=YT$. Let $\angle PST=\alpha$. Then, $\angle TSX=90-\alpha$, so $\angle SXO=\angle TXO=\alpha,$ so $\angle XTS=90-\alpha$. Then, $\angle PTO=90-\alpha.$ Thus, $TS$ and $TO$ are isogonal in $\triangle TFX$, so $\angle FTS=\angle XTO$. However, if we let $E$ and $F$ be the feet from $O$ to $PT$ and $ST$, then from $FOET$ cyclic, $\angle YOX=\angle FTS$. Thus, $\angle XTO=\angle YOX$, so $(OTX)$ is tangent to $OY$, thus $OY=\sqrt{YX\cdot YT}$ so the locus of $O$ is the circle centered at $Y$ with radius $\sqrt{YX\cdot YT}.$

We present a motivated solution (what's spiral similarity?). Let $\measuredangle$ denote directed angles modulo $180^\circ$.

We want to find a necessary and sufficient condition for $O$ that implies $\overline{PS}$ is tangent to $\omega$. This is equivalent to $\measuredangle TXS=\measuredangle TOP$, which is equivalent to $\measuredangle TXO=\measuredangle TOY$. Notice that this is exactly the condition for $\overline{OY}$ to be tangent to the circumcircle of $OTX$, so $YO^2=YT \cdot YX$. Thus, it is equivalent for $O$ to lie on the circle centered at $Y$ with radius $\sqrt{YT \cdot YX}$. Other than the points on $\overline{XY}$, all choices of $O$ on this circle create a nondegenerate configuration, so this is the locus of $O$.

Let $Q = PS \cap \Omega$, and $M = TS \cap (ABC)$, which is evidently the bottom point. Consider the two spiral similarities at $T$: $S_T: \triangle TMQ \mapsto \triangle TPS \implies \triangle TOY \sim \triangle TPM$ $S_T: \triangle TOP \mapsto \triangle TXS \implies \triangle TOX \sim \triangle TPS$ These imply $\triangle YOX$ is similar to $\triangle MPS$, so angle chasing gives \[\angle YOX = \angle SPM = \angle MTP = \angle YTO \implies \triangle YOX \sim \triangle TYO,\] implying the length condition $YO^2 = YX \cdot YT$, which is fixed. Thus the locus of $O$ is a circle centered at $Y$ with radius $\sqrt{YX \cdot YT}$, but it can be seen that it must exclude the points on line $XY$. $\blacksquare$

$ $ I thought the hint meant to look at the two tangents from $P$ ;-;