Yes, if you -replace- a skill, you lose the skillpoints you spent in that skill, because that skill is now gone. But if you are merely -adding- a new skill (i.e., training a higher rank skill), then you are not losing any lower rank skills.
In less confusing terms, there is no skillpoint 're-spec' or shuffle for Bridge Officers. If you replace a skill you have already trained (which gives you a different skill in its place, with no skillpoints invested into it), the only way to train that new skill up is to get more Bridge Officer points. This is due to the fact that you can always earn Bridge Officer skillpoints, so you can throw them away however you like.
I am almost certain I swapped out my tactical officer's FIRE AT WILL 1 ability with TORPEDO HIGH YIELD 1 and the points I put in FIRE AT WILL were still there for the new abillity. Are you sure of this?