Try it out. Setup dnsmasq and connect your phone to the network. You'll see a ton of requests initially, that gives you some idea of what apps/services/accounts are on the phone. Let the phone go to sleep, and watch what is sending requests in the background. Many services use very specific host names which indicate what is being processed.

On the TV, it would be similar. You walk into the room and it starts sending packets? You say something unrelated to its trigger word yet Wireshark shows activity? Suspicious. If you can get a certificate onto the TV you can use mitmproxy to view the HTTPS traffic, but that's probably kinda difficult.

I do not use smart TVs but I have been doing stuff like the above for a while. If they are recording and storing stuff some engineer eventually figures out, it's not an NSA backdoor.

I'm not saying they are/aren't, I do not know, it just seems very unlikely and improbable especially given smart phone ubiquity. What is known to be actually occuring is a complete violation of consumer privacy for marketing purposes, but OPs form of spying is so far unsubstantiated.

Now, can that TV be hacked and used by your neighbor to spy on you? Or can your government access your mic/camera? That's an entirely different question and field of expertise.

More info