Where are you when this happens?

You might just leave the queue window open and see if the invite appears in there. I think there are still some cases where you don't get the notification popup like you should. I had that happen once last night while zoning -- my group got the queue but I didn't. I opened the PvP queue window and just had to click "join". Apparently zoning (or something else) caused me to not get the popup.