their are good fleets out their, I have a few characters that are not in a fleet and I know what you mean, when im on them I get a bunch of random invites too, but I believe these are not the larger more structured fleets that you would actual want to join. I think most of the random invites are from new players trying to make their own fleet and get as many people as possible because they dont understand in a game like this its quality over quantity.
So its not people being to faced or anything, most likely the people who pass you up (dont know what you did to **** people off but not important) are the fleets that are more organized and use the forums or their own website. People who send you random invites most likely dont use the forums and just invite random people so complaining on here wount change anything. My advise, block fleet invites or do what some else said, join a small fleet called 'leave me along' just to stop the invites.