Most of them were probably just quiet racist beliefs at home and in implicit ways in public.

It's easier to miss, but it's also easier to retreat from, since it's not such a public belief.

Just like most people weren't civil rights activists, most also weren't frothing rally style racists.