The ambient libs mostly don't care as much about politics or have checked out partially, they may be outraged over some blatant nazism but they aren't organized or mobilized to do anything about it and will quickly fall back into their malaise.

When the Nazi/fascist minoritarians take power, they usually have the blatant support of around 30% of the population. The other 70% aren't all anti-fascist partisans or Schindler's hiding and saving jews. Most of them are ambient libs who continue being ambient libs/ambient fash and just acclimatize to the the current political zeitgeist.