Nazi instances will proliferate and it benefits nobody else to stay in federation with them. It makes the whole fediverse less usable and more dangerous. And whether you like it or not it sends a message to people who are targeted by them that they are not truly welcome here, regardless of whatever moderation rules are espoused.
And in North America, as in many places, these people are acting as a propaganda arm for a literal violent terror movement. Sometimes under a fig leaf of ”irony” but it makes no material difference whether they’re chuckling when they spew shit to me