They did sort of use CO2/CO in the earlier days, in the form of poorly maintained van trucks that would drive around the camps with their exhaust piped directly into the back of the van. That was slow and inefficient though, as the truck had to run long enough to actively displace all the oxygen within the volume of the van.

Zyklon B from its Wikipedia page was intended as a pesticide so there was already a large industrial supply lying around. It works on a cellular level and will cause widespread cell death within 2 minutes of inhalation at extremely low concentrations, making it ultimately much faster with less maintenance.

I'm not a historical expert, but the deaths caused by Zyklon B are also described to be much, much more violent than simply passing out from oxygen deprivation- and I could posit that the Nazis, viewing jews et al. as not human, would have preferred such a death for them.