Same, generally speaking when I'm writing a comment in order to disagree with someone I want that other person's comment to be more visible to other readers. That way they can read it, see my response, and realize how wrong the original comment was and how right I am. :) I save my downvotes for comments that are so wrong they're not worth a response.
I'll even sometimes downvote a comment, ponder for a moment, and then remove my downvote and write a response instead.