To try to answer, succinctly (which I'm bad at): looking backward is easier than looking forward. What I mean by that is since you didn't get into the series until 3, it makes sense that you wouldn't have a problem with 3 and 4, since it's harder to see what the series could have been...as pretentious as that sounds.

Where much of the hate comes from (and I think a lot of it is overblown - I'm not trying to justify the behavior of the maniacs out there) is that the overarching progression of the series feels reset. Fallout 1 -> Fallout 2 showed a progression in a *post-*post-apocalyptic world, with society advancing again, to some degree. Shady Sands grew between 1 and 2, and was the foundation of the NCR.

So Fallout 3 at the time was IMHO a disappointment because the setting felt more generic, and like they were just playing the greatest hits from 1 and 2. I get the arguments that the setting in-universe was hit harder, but it still felt weird that it was post-apocalpytic instead of post-post-apocalyptic.

One reason (as always, IMHO) that New Vegas was so popular is that it continued to build on 1 and 2. We saw the NCR had continued to grow, other factions rise in importance, and generally felt less like the bombs had dropped the year prior. It's what a lot of folks hoped Fallout 3 would be, in that sense. That's my own biased view though, so take it with a grain of salt - there's folks who want more humor, only isometric, more complex and branching storylines, etc.