The message is “If you disagree with me, you will be banned”

It used to be that votes were meant to be used as an indicator of the quality of the post according to the community guidelines, not how "agreeable" a comment or post is. This cultural change is one the most toxic behaviors that made Reddit such a crappy place for discussion.

This was already bad on Reddit, but at least there one could avoid this problem because people were used to browse only the subreddits they subscribed to, so niche subreddits could still have some semblance of "good" community participation. On Lemmy, most people browse by /all and lots of them still treat the downvote button as a some mechanism to train an algorithm. These users are the worst.

In the beginning, I was actually sending DMs to people asking them to please not downvote something if they were not part of the community and their reaction was basically "I don't want to see this, so I will downvote to bury it" (completely ignoring the fact that they could simply hide the post or stop browsing by /all).

So, while "banning everyone who downvotes the post" might seem an overreaction, I could definitely see a moderator could flag a vote as coming from a non-community member and use that flag to ignore their votes in the ranking systems, and I would love to have a bot that auto-messages every clueless downvoter explaining the proper netiquette around votes for non-community members.