I was for disabling down votes the last goaround wrt this topic, and I'm of the same opinion.
LemmyNSFW hasn't grown much more in the past year and this will just drive away more posters (of which there aren't much left) imho
Additional - I'd prefer that others block me (so they don't have to see what I post) or block/don't subscribe to various communities I mod that they don't want to see as part of their feed, rather than have them downvote individual top level posts.
Enabling downvotes wrt comments, I'm okay with...
Very well articulated π, especially your last point, given the still fragile growing-pains state that LemmyNSFW is currently in