I have no advice, but I have a similar issue. Firefox on Windows 10 with just uBlock Origin.

YouTube works fine at first, but if I watch a lot of videos in a row then after an hour or so it starts to load slowly. Then it stutters, then finally I get audio and a single frame for the whole video. Shorts seem to accelerate the descent. I haven't tried live streams yet, but I bet I'd get the same result.

Completely closing all Firefox instances and reopening fixes it for an hour. It acts like a memory leak, but I'm not even close to maxing out my RAM when it crashes. My fans get loud which makes me wonder if it's botching the hardware acceleration.

Sadly I can't help beyond telling you you're not crazy.