I ignored the Klingons and destroyed the Doomsday Device, the mission ended fine. You could have run into one of those "mission not completing" bugs.
"Don't let them promote you. Don't let them transfer you. Don't let them do anything that takes you off the bridge of that ship, because while you're there... you can make a difference." - James T. Kirk
What has helped me is warping out and then right back into the same sector. Most times it goes ahead and the ending text pops up. Not all the time, though. Just one more thing that needs to be worked on.
Probably the escort that has to be destroyed (it is a game objective). If you attack the Targ first and bring it low on HP, you can board it and forget about the escort. Since you cannot go back in mission progress, I regard this as bad mission design.