I can see the logic there, but why not vote based on relevance rather than agreement? That way comments that are on-topic and further the conversation rise to the top, regardless of whether they align with the Lemmy hive-mind. Some of the best threads are the long ones with a spirited back and forth between ideological opposites, and those would go away (or be pushed to the bottom) if both sides simply down-voted each other back to net-zero.

As a weird byproduct, we also get fun stuff like Hanlon's Law, which states that the fastest way to find the correct answer to something online is to confidently state the wrong one on Reddit/Lemmy and wait for your comment and the actual answer to float to the top. After all, people love to correct one another, and we often come to Lemmy to learn about other points of view and have our own views challenged. As long as everyone is debating in good faith and trying to add value to the conversation (which should be enforced by downvote), differing opinions are a good thing.