If you do any of the colonization chains in the star cluster, they all require colonists which are 'lost' when completed. If anything, I find there is a shortage of colonists. In each chain, there are several of them that require between 3 and 5 colonists. Multiple that by 14 star clusters, and it's a lot of colonists that is needed to complete them. The short answer is that I NEVER run the Relocate colonists, and instead, save them for the colonial chains.
The colonists primary use is for the colonization chains. I know you are playing the doff system casually, but a quick look at doffjobs spreadsheet can tell you where to employ your colonists without wasting them in relocate missions. if it really starts to be a problem, just stop doing resettle missions.