@billothekid2 I'm with @FaceDeer on this, it's way better to know you have a Downvote Fairy than to just think no one appreciates your comments.
Back on reddit it happens a lot, but the targets are more likely to feel discouraged or think the person they are replying to is the one downvoting them.
Besides, I had someone doing this to me here on kbin for a couple of days and they are not someone I ever downvoted or had even argued with.
I asked them to stop and they downvoted me one last time and then stopped. I don't think they'd realized I can see them.
So the cause isn't that people can see your downvotes, it's that some people are just dicks.