I mean it's the whole tolerant paradox right.

When you view things in the context of a social agreement, there is no longer any paradox.

If these people have broken the social agreement to be tolerant, they have then intentionally and explicitly removed themselves from that agreement, thereby opening themselves up to intolerance thanks to their intentional and explicit rejection of said tolerance.

It’s much the same way as outlawing worked in the old days - in the absence of a police force, you willingly agreed to follow laws that had been laid down. If you openly broke those laws in clear defiance of them, you could be removed from their protections. Ergo, you became “outside the law”, allowing anyone to harm or even kill you without legal censure.

Because if you clearly don’t want to be a part of an agreement, why should you have any right to benefit from it’s protections?