aren't you still limited by ambient air temp because the hot side of the Peltier needs to be cooled by air anyway?

No, that's why the baby fridge works. The peltier TEC in the fridge can produce temperatures below ambient, but generally not below freezing. Computer chip TEC coolers would go farther and push more power through the TEC and do things like water-cooling the hot side instead of a little fan.

The reason they don't get used anymore more complicated. It's my understanding that basically TECs have a sort of limit on the amount of heat they can push from the cold side to the hot side, because moving more heat means more energy used and thus more waste heat. Apparently most modern chips are past that limit. IIRC, TECs can only move something like 100w of heat - past that and they start to heat themselves up because of waste heat. Modern chips can be like 300w.

Sub ambient cooling also comes with a bunch of issues like condensation, so no one really uses it day to day. Also, chips are run so close to their limits now that cooling like that doesn't get you nearly the performance difference it once did.