It really depends on which parts of the series you enjoyed the most, I guess. The original and New Vegas had that more somber, serious tone, and a more contiguous wasteland that sought to evoke a specific feel and a cohesive whole. Fallout 2 and 3 were gonzo outings and proud, patchwork theme-park quilts that took things in bizarre directions or transplanted bits directly from the pop-culture well rather than filtering them through that bleak, understated, gallows-humored detente-era lens. "What else can we throw in that might be fun for the players? TALKING deathclaws! Yakuza! Mobsters! Duelling superheroes! Wait-- what about aliens?"
I truly don't mean to come off as derisive, and I know a lot of people did enjoy those things. I did myself, to an extent. I'm just one of those who prefers his wasteland to take itself a bit more seriously, to attempt to cast a lens on the human condition and prioritize story over spectacle, lore over day-to-day looting and adventuring. The humor and the gonzo bits should be gilding on the framework, not an actual functional part of the apparatus, imo, no matter how small.
Aliens don't necessarily hurt anything just by existing, but what do they contribute? If you're not tying them into the narrative or the world (which you seem to agree is a bad move) they're really just there to say "hey, look at this thing we can throw in," and that's a very slippery slope. It's taking style over substance, ignoring form and function in favor of flash. You could use the "50s pulp" argument to justify a lot of things. Why not a flying car, or Enclave jetpack commandos? A second wave of the Chinese invasion staged from a secret base they built on the moon before the war went hot? Fallout 5: Invasion of the Lizard-Riding Mole-Men? There's a balance that needs to be kept.
Again, this is all just my take, and I apologize if I come off as strident. I'm just attempting to use a bit of over-emphasis to better answer your question. At the base of it you're right, and if living aliens became established canon (Bethsoft's stance on DLC canonicity is as-yet unclear) but the devs were smart enough to keep them from being anything but a marginal presence in the games, my ilk would probably grumble and hate it and then simply pretend it never happened. Once something's in the world, though, there's really no escaping that it's a part of it. Or, for that matter, the looming threat that it will start creeping in by slow degrees, become more and more normalized and expected until someone forgets where to draw the line of good taste. People who have been with the franchise since the beginning have seen it happen again and again, so I hope you'll concede it does make sense that some of us are so wary.