Check if your TV is playing the surround track. Modern TVs are hell-bent on serving you the multi channel audio, regardless of it's being played through the built in stereo only speakers.
Surround is mixed for high dynamic, talking is 'talking volume' and loud shit is indeed loud. In contrast stereo is pretty much normalized, so you can here everything at moderate volume.