There are a couple important things to consider though.
What is it we are really asking when we talk about “ first time home-buyer?” (FTHB)
Is it a negative thing because FTHB’s are riskier? Or is it a positive thing since there are ostensibly programs available only to FTHB’s?
It’s actually only ever a negative thing because FHA, VA, My Community and Home Possible are all loans that many people think are available only to FTHB’s, but they are available to repeat buyers as well. Seriously, I don’t know of any loan programs that are only available to FTHB’s that are not also available to repeat buyers.
Of far far greater importance is whether or not the buyer owns any other property and whether or not the new loan can be considered “ owner occupied.”
Some programs that are traditionally considered FTHB programs will not work if you own other real-estate, so those are out. But that doesn’t matter nearly as much as “owner-occupancy.”
If you own three houses and are re-locating to live in the new house, then you are still “owner occupied”. You will have access to the same great rates available on FHA and other programs normally considered to be FTHB only.
If you just moved across a border, not far from your last house, the lender might consider you to be “non-owner occupied.” In other words, they will think you are buying another investment property since you already own 3. In that case, your rates will be higher.