im pretty sure when it comes to sjw specifically its down to the @MomoTimeToDue drama, the MeanwhileOnGrad comm with the racist banner, and extreme transphobia from some users (eg a trans person on hexbear was PM'd saying they were mutilating themselves and children)

I have a different answer than blakeus12

the two factors are

1. An ambient sense of how unreceptive the instance is toward us. If the distribution of responses we get is skewed too far toward "I don't give a fuck what you have to say, I'm not reading that, fuck you you disgusting tankie," that's a reason to not federate. And I mean... I'll be the first one to admit hexbears can be annoying and too eager to dunk on people, but on the flip side I also think our views are often misrepresented and demonized, and it's frustrating when people won't even bother to understand what we actually think or why.

2. If any of our users feel harrassed or unsafe, even if the culprits are a small minority of the instance, we are liable to defederate. And this shouldn't be taken personally! We all know lemmy still has limited mod tools. But we prioritize each other over federation, we're a close-knit community who have been together for years.