It's absolutely ableist. My ADHD makes me worse at games for sure (and also way less able to tolerate punishing design such as the Dark Souls corpse runs), also some things like tapping rapidly I just can't do at all. The only answer gamer bros have for me is "git gud" or its even worse evolution "skill issue".
It really triggers the RSD sometimes but what can you do, I just play easier slop now (especially since work has crushed my soul) and have more fun. I would actually love a website that lists the bullshit in each game like infrequent checkpoints, difficulty spikes, lack of quest markers etc so I can know to avoid them.
I can picture a type of person that would enjoy Elden Ring for its artistic value but suck at games due to reasons outside their control (bad reaction times, poor concentration, sensory processing difficulties etc) or even due to not having enough time to "git gud". Why shouldn't they have their easy mode? Other mediums aren't that inaccessible to people