Speaking as a story-heavy RPG enthusiast, so my focus is more on story-telling and exploration. I don't think it's the graphics that's holding back the exploration potential, but rather the complexity of actually creating huge game worlds. You tend to end up with either a procedural generated world without a lot of cohesion, or one that's a mile wide but an inch deep in interactivity.
Just look at Baldur's Gate 3. It's a hugely complex and reactive game world, but it's locations and the way you are allowed to explore them are reduced to three linear chapters. Even if you switched to, say Baldur's Gate 2 era graphics, it would still not be possible to create a game in a single huge explorable world with the same level of complexity and story telling.

Though I'm definitely with you on scaling down the graphics in exchange for richer and more interactive worlds, I do think there's a hard limit on how much better those worlds would get.