We claim that the answer is all $a \ge 2019.$ Call a positive integer $n$ $\mathbf{good}$ if $2019 | s_{4038}(n)$, where $s_{4038}(n)$ denotes the sum of the digits of $n$ in base-$4038.$ Firstly, we'll show that all $a < 2019$ do not work. It's trivial that $a \le 0$ don't work, so suppose that $0 < a < 2019$ is a real number. Assume, for contradiction, that there existed such a sequence as given above. Then, there would exist a positive integer $C$ so that for all $m \in \mathbb{N}$, there are at least $\frac{m}{a} - C$ good integers in $[1, m].$ However, this is clearly bad since we can easily note that $[0, 4038^k-1]$ contains equal numbers of every residue class mod $2019$, in terms of the sums of digits of the numbers in $[0, 4038^k-1]$ in base $4038$, for all $k \in \mathbb{N}.$ Now, when $a \ge 2019$, we can simply take $a_1 = 1$, and then let $a_{2k}$ be the unique positive integer in $[4038k, 4038k+2018]$ which is good and $a_{2k+1}$ be the unique positive integer in $[4038k+2019, 4038k+4037]$ which is good for all $k \in \mathbb{N}.$ $\square$

The answer is all $a<2019$. We will call a positive integer good if its sum of digits in base $4038$ is divisble by $2019$, and bad otherwise. Consider the intervals $$I_k=[4038k,4038k+4037]$$Obviously $I_1\setminus{0}$ contains $1$ good number which is $2019$. CLAIM 1. For all $k\geq 1$, $I_k$ contains exactly $2$ good numbers. Proof. Let $s(n)$ be the sum of digit of an integer $n$ in base $4038$. Let $s(k)$ be $S$, Suppose $a=4038k+b$ is an element of $I_k$. Then $$s(a)=S+s(b)=S+b$$Hence $b\equiv -S\pmod{2019}$, since $0\leq b\leq 4037$ there are exactly two possibilities. $\blacksquare$ Now, let $x_k,y_k$ be the two good integers in the interval $I_k$. Firstly we will construct a counter-example for $a=2019$. Indeed, it suffices to take $$1,2,2019,x_2,y_2,x_3,y_3,...$$ Now suppose $a<2019$. Let $f(n)$ be the number of good numbers less than or equal to $n$, and let $B(n)$ be the number of bad numbers among $a_1,a_2,...$. CLAIM 2. For every positive integer $k\geq 2$ we have $$f(4038k)<2k$$Proof. This easily follows from CLAIM 1. $\blacksquare$ Now, since $a_n\leq an$, there are at most $f(an)$ good numbers among $a_1,a_2,...,a_n$. Let $k$ be the integer such that $4038(k-1)\leq an\leq 4038k$, then we have $$B(n)\geq n-f(an)\geq n-f(4038k)>n-2k\geq n-\frac{an}{2019}-2=n\left(1-\frac{a}{2019}\right)-2$$Since $1-\frac{a}{2019}>0$ we have that $B(n)$ is unbounded, this completes the proof.

We claim the answer is $1\leq a<2019$. Note trivially $a\geq 1$, as $a_1$ needs to be a natural number. Define $f(n)$ to be the sum of the base-4038 digits of $n$. We consider the sequence $s_k$ such that $s_1=2019$ and for each $n\geq 1$, $s_{n+1}$ is the smallest integer greater than $s_n$ such that $f(s_{n+1})\equiv0\pmod{2019}$. Note that in the numbers $2019k,2019k+1,\ldots,2019k+2018$, they only differ in the last digit, so $f(2019k),f(2019k+1),\ldots,f(2019k+2018)$ is a residue class modulo 2019 and thus must contain exactly one number such that $f(n)=0$. Thus, we get that $2019n\leq s_n\leq 2019(n+1)$, implying the sequence $1,s_1,s_2,s_3,\ldots$ disobeys the constraints for $a\geq 2019$. For $a<2019$, suppose we found such a sequence such that only a finite number of $a_1,a_2,a_3,\ldots$ satisfied $f(a_n)\not\equiv 0\pmod n$. Thus, for some $m>0$, for all $n\geq m$, $f(a_n)\equiv 0\pmod{2019}$ - in particular, $a_n=s_{b_n}$ for some sequence $b_n$. Now, we can easily note that $an\geq a_n\geq 2019b_n$ for all $n\geq m$. In particular, this murders the problem because $b_{n+1}>b_n$, so $b_{n+1}\geq b_n+1$. Thus, we have \[a(m+(n-m))\geq 2019b_n\geq 2019(b_m+n-m)\implies \frac{a_m-2019b_m}{2019-a}\leq n-m\]which is absurd.