The spying that's openly admitted in terms and conditions should be alarming enough — if anyone actually read and understood all the legalese. Consider this: https://time.com/5568815/amazon-workers-listen-to-alexa/

I've seen Android phones activate Google Assistant seemingly at random many many many times. They're only supposed to activate when called by a specific phrase like "okay Google", but there are plenty of false positives, and every time that happens, an audio recording gets sent to Google. Same deal with Alexa and Siri. This is, of course, allowed by the terms and conditions.

At least Android makes it visible to the user when this happens. I wouldn't bet on smart TVs doing the same.

At this point there's not much you can do about it. Even if I secure my own devices and my own home network, that all goes out the window the second anyone else walks in my door with their own smartphone.

That said, I agree that the claim is likely false with third-party apps on modern smartphones from major brands. It's not easy for background activities to access the camera or microphone without the user's knowledge on iOS or Android. First-party and second-party spying is hard to avoid, though.