Return to office has nothing to do with the people who do better in an office environment. It's about forcing those who don't do as well in an office, and those who the commute or time away from the house isn't feasible, back into the office.

Sometimes that "chance encounter" is really just one mouthy coworker cornering another coworker who doesn't like saying no even though they have work they need to attend to. For those people, just avoiding chatty coworkers is a challenge and they get less done because people won't leave them alone with tangential bullshit.

Some of your coworkers have sensory issues that are constantly irritated and distracted by being in a loud, bright, and chaotic office.

This isn't about you. You were always allowed to be in the office (save for pandemic lockdowns). So don't make it about you.