I don't see an issue here.

1. try to explain your downvotes, or, better yet, voice your disagreement and have a discussion without relying on downvotes to express yourself. Use your words.

2. when you feel it necessary and that using your words isn't working, upvote/boost everyone but the person in question. Realize downvotes don't really do what you'd hoped and are a poor crutch and means of expression. Elevate what you do agree with. Worry less about what you don't.

3. just don't care about internet points. As others have said and covered extensively.

Hopefully, what remains is that conversations are important. Having an outlet for undescribed/unformed/ambiguous disagreement/dissent really doesn't add anything to anything for anyone. Making a point or saying something correct is more valuable than shooting down those you find to be incorrect. And, ultimately, it's not just what you think that matters. The consensus of the group and the conversation that gets them there, the experience and the interchange and the community itself are all much more valuable than the destination of a single conversation.

Edit: (And, for the record, while I disagree with your assessment, I've boosted and upvoted your thread, because I think the conversations being had here are valuable and worth seeing by more people.)