Before the Challenge System as introduced, creating a private queue with some friends revealed this bug as well. Experimenting, what we learned was if the host hit start too soon after a player joined, they were sent to the map but where not technically in the queue and as such kicked out. We noted they never did show up in the scoreboard either.
BTW the other associated bug is even though you've been booted the server "thinks" you are still in the queue even though you can't see/change that status in the menu. Over the weekend I had no way to get out of the queue, and was continually being sucked in then immediately booted.