I also recently got a set of those. I've really been enjoying them for at the desk and on the go.

The challenge you have is that a device that meets your spec is going to have to decode the optical surround sound signal from the TV which means it is going to need a chip and support for the codecs such as Dolby Digital and DTS. That is going to drive up the price and why what you're seeing are AV receivers. Unfortunately you don't need a switch, you need a surround sound DAC and those are not very common.