I think there's something to be said for it being public. If someone's downvoting all of your content for no reason without engaging with it, that's obviously not someone worth your time and it may be a decent idea to just block them. I could also imagine some communities making it explicitly against the rules to downvote constructive comments for no reason, for instance.
At any rate, my understanding is that the actions must be at least publicly accessible in order for federation to work, so the only thing that Kbin could do is simply not openly display that data. Perhaps making it less accessible would reduce the temptation to look, but it'll always be available to anyone who truly wants to see.