I think it's mostly cowardice, personally. People don't want to risk putting their own choices into a game based entirely on choices, just in case they aren't as good.

It's work and effort. D&D seems to have evolved from a game of random tables and strategy that required constant improvisation from the DM, to a game of (open ended) story telling. Character death seems to be less of an option.

So now DMs think they need to build a complex story, while allowing players to make choices. They need to prepare challenging encounters that aren't too challenging. And D&D combat tends to repetition, so they need to find ways to spice it up.

The handful of WotC modules that I've seen don't support that. The module has a single path and a bunch of dull combat encounters.

A lot of people don't know how to fill the gaps, and WotC has never bothered teaching them how.

This. I'm reading the Cyberpunk RED sourcebook now. I really appreciate the GM hints and suggestions. I don't remember anything like that in the 5e DMG.

Any rules they did get are rules of thumb and aren't something to use without thought (like CR), so people complain for reason 1 again.

Worse, some rules/features are holdovers from previous versions that don't make sense in the current game.