It should only be able to be disabled if combat is over in PVE mode only. That's one way to balance it in my opinion.
That's exactly what it does right now. Currently, whenever you're at Red Alert is the time the game considers you to be in 'combat'. To prevent people from going to warp or full impulse the second they're not being fired on, it waits a little bit to make sure you aren't just temporary out of line of sight of your opponent or just jamming sensors or something.
Being able to manually cancel red alert isn't going to happen, unless they untie it from being in combat (but then they'd have to use something else to let us know when we are and aren't in combat). The better request would be to ask them to look at the length of time the game waits before dropping you out of combat, which would have the added benefit of ending red alert sooner.