yeah this is definitely a big concern with smaller instances, there are a couple of tricks to this general problem from what i've thought/seen of over time.

The obvious one is a democratic vote, literally just ask people in the instance, the second obvious one is to vet people in that instance specifically and personally. And if they cause problems just yeet em. You're the dictator after all. The most common option is to have a decentralized moderation team made up from the general community, which is extremely common and generally works, though suffers from the opposite problem, ironically.

I think if i had to moderate a lemmy instance i'd probably do a mix of heavier vetting (although most of it would likely be after they initially joined, a vibe check i suppose. As well as just being a literal direct dictator, depending on the size i might have "chaos control" mods, just to keep goofy shit from happening while i'm away, or to provide some support, who knows. And naturally, i'd focus on community votes, i'd be curious what the community instance itself had to say.