Raise the Hammer
The purpose of comment scoring is to encourage and reinforce civility, respect, and intellectual honesty in commenting.
By Ryan McGreal
Published May 11, 2009
Raise the Hammer is an online community of choice dedicated to urban revitalization and sustainable economic development in Hamilton, Ontario. One important form of participation on the site is in the comments on posted articles and blog entries.
Due to a recent escalation in the incidence and disruptive influence of "trolls" - comments posted for the express purpose of disrupting and undermining the discussion. One result has been that many RTH readers have become reluctant to: a) post comments for fear of getting trolled; or b) bother reading the comments as they increasingly constitute fruitless debates with a troll.
As a result, RTH proposed implementing a system of community-based comment moderation based on votes by registered users. The overwhelming response we received, both in the comments on that post and in direct correspondence, has been enthusiastic support for this system of user moderation.
However, we also received some comments that expressed concerns about a) suppressing free expression and b) RTH users 'gaming' the system with multiple sock-puppet accounts. The design of the comment moderating system attempts to address these valid concerns; and it will be a work in progress, adapting if and when problems emerge.
Here is how the system will work:
Registered users will be able to vote on comments - a simple vote up or down on each comment. You can always change your vote after the fact. Clicking the same arrow again cancels the vote, and clicking opposite arrow reverses the vote.
Registered users will have an option to set a comment score threshold - a score below which comments will be hidden by default. The default is to show all comments, and even hidden comments can be revealed by clicking a button on the hidden comment.
Registered users will also have an option to turn comment voting off entirely, so they only see the comments themselves, not their scores.
No registered user will be able to change the settings for what another user can see. Each user will choose their own settings in their user profile.
Anonymous users will still be able to leave anonymous comments and will see all posted comments (regardless of score) plus comment scores, but will not be able to vote. To do that, they will have to register a user account.
The purpose of comment scoring is not to downvote unpopular opinions but to encourage and reinforce civility, respect, and intellectual honesty in commenting. Toward that end, we propose the following guidelines:
Please DO:
Post comments that contribute meaningfully to the discussion. Good comments: add new information, correct misinformation, offer a new perspective on an issue, challenge a conclusion already drawn, or propose a resolution to an existing debate.
Upvote comments that are well-researched and reasonably argued, regardless of whether you agree with the conclusion or how you feel about the person who wrote it. If someone known for trolling posts a reasonable comment, encourage more reasonable comments with an upvote instead of dismissing or punishing it because of who wrote it.
Downvote inappropriate comments. Inappropriate comments: use rude or insulting language, are needlessly inflammatory, seek to provoke an emotional reaction from others, are attempts to disrupt and derail the discussion, are offtopic or irrelevant, or abuse evidence and reasoning to defend an unjustifiable conclusion.
Write in defence of a comment you feel is being downvoted unfairly just because it draws an unpopular conclusion or was written by someone controversial.
Please DO NOT:
Do not post inappropriate comments that: use rude or insulting language, are needlessly inflammatory, seek to provoke an emotional reaction from others, are attempts to disrupt and derail the discussion, are offtopic or irrelevant, or abuse evidence and reasoning to defend an unjustifiable conclusion.
Do not post comments just to agree or disagree with another comment. "Me too" posts don't add to the conversation and make threads long and awkward to navigate.
Do not post comments to complain about being downvoted. It's off-topic and disrupts the conversation.
Do not bait people to downvote you for posting an inflammatory comment.
Do not post comments to complain that an article or comment is "biased" because it promotes an urban agenda. That's the whole point of this site.
Do not post multiple consecutive comments. It's rude to monopolize the comment area and crowd out other potential commenters.
Do not vote based on whether you agree with the author's opinion. Comments should be rated for the quality of their argument, not which side they've taken.
Do not vote based on who wrote a comment. Try to judge each comment on its own merit.
Do not create a phony "sock puppet" RTH account just to add upvotes for your comments or downvotes for someone else's comments.
It is up to all of us as members of an online community to reflect and reinforce our community standards of respect, civility and intellectual curiosity in our exchanges with each other.
We all have a responsibility to avoid using argumentative fallacies, to maintain a civil and respectful tone with each other, to make a sincere attempt to contribute meaningfully to the discussion, and to encourage others in the same responsible behaviour.
By reuben (registered) - website | Posted May 11, 2009 at 14:48:25
im very interested in seeing how this works out. nice work getting this up and running so quickly.
To date, there's 30 comments made (all at least 2 days old) on this RTH post asking users their opinion of the comment-voting idea: http://raisethehammer.org/blog.asp?id=13...
I don't know if that counts for you, but that's a fairly substantial effort (and response) in my eyes regarding user input.
RTH staff: Thanks for getting this in so quickly and seamlessly -- sometimes these discussions can drag on for weeks, or implementation can be messy. This looks quick and self-explanatory.
By Brioski8 (registered) | Posted May 11, 2009 at 18:41:35
Thanks for this Ryan and RTH staff. You inspired me to register finally. As we can all see, the comment system is working already. SMZ's poorly researched and perhaps hasty comment has the negative score it deserves.
By nobrainer (registered) | Posted May 11, 2009 at 21:50:01
Here here! RTH is nothing if not responsive to it's readers/commenters. I'm not sure how complicated the voting is to put in place but kudos for doing it so fast.
Next up: voting on articles themselves...
By jason (registered) | Posted May 11, 2009 at 21:59:57
good job Ryan. Should be interesting to see how this works.
By smz180 (registered) | Posted May 12, 2009 at 09:38:25
Yup, I read that post. If all opinions voiced we're so valued, I'd really like to know which additition functionality was added after that dialog and which user comments they were based on?
My opinion getting "bad ratings" based some assumptions of what I've read is half of why I'm posting this to begin with. Just because an opinion is contrary to most users doesn't mean it's less valid.
Lastly, for those who voted the dreaded "Negative". Exactly where does my comment fail in on the Comment Voting Etiquette above?
By reuben (registered) - website | Posted May 12, 2009 at 10:08:22
i agree, smz180, the negative ratings on your comments are unwarranted. especially the negative votes on your followup comment questioning why.
this is not a good way to start with this new system, everyone! i dont think smz180 was trolling, just stating an opinion.
By reuben (registered) - website | Posted May 12, 2009 at 10:15:22
i had one note on the etiquette list above:
DO NOT: Post comments just to say you agree with another a comment. Use the upvote arrow for that. DO NOT: Downvote a comment just because you disagree with it.
are these not conflicting points? if agreement earns a vote up, shouldnt that be balanced by disagreement earning a vote down? in my opinion, voting should never be used for agreement or disagreement of what is being said.
By z jones (registered) | Posted May 12, 2009 at 10:18:29
Well, I registered an account, so thanks for making it worth my while.
@smz180 -
I downvoted your first comment which basically accused the site admins of lying. And then I upvoted your second comment which asked a fair question and pointed out a problem with the voting.
This is an important issue that the users can deal with by encoraging responsible voting on each other. I bet you'll see people vote one way, then read a reply and reverse there vote, that's what I see on other sites that have comment voting.
Also, don't feed the trolls! I've been saying this for awhile now but now instead of feeding the troll you can just downvote and move on.
By UrbanRenaissance (registered) | Posted May 12, 2009 at 10:41:26
Ryan wrote: "I'll try to think of better wording - and please feel free to make a suggestion if you have anything in mind."
How about...
DO NOT: Post comments just to say you agree/disagree with another comment. "Me too" posts don't add to the conversation and make threads long and awkward to navigate. DO NOT: Vote on comments based on the author's opinion. Comments should be rated for the quality of their argument, not which side they've taken.
By UrbanRenaissance (registered) | Posted May 12, 2009 at 11:04:01
One more small thing Ryan, in the "DO NOTs" section the change in item c now makes item d redundant.
Though I suppose you could always replace it with
DO NOT: Nitpick the Editor
:)
By reuben (registered) - website | Posted May 12, 2009 at 11:09:14
I bet you'll see people vote one way, then read a reply and reverse there vote, that's what I see on other sites that have comment voting.
good point zjones. i think that this could happen a lot, and an example of where voting a comment down will encourage the original poster to clarify or re-frame their argument.
By Blossomful (anonymous) | Posted May 12, 2009 at 11:12:52
"Next up: voting on articles themselves"
Seconded. It's only fair...
By jason (registered) | Posted May 12, 2009 at 11:40:15
sure. go for it. Then we'll see which ones of us deserve a raise!
By UrbanRenaissance (registered) | Posted May 12, 2009 at 11:43:43
I don't think voting on the articles themselves is necessary, or even appropriate.
If the voting is being used as its intended, (to downgrade low quality posts and upgrade high quality posts) then Ryan, as editor is already acting to prevent low quality articles from being added to the site.
If we agree or disagree with an article then we have the comments section to argue all sides of the issue.
By reuben (registered) - website | Posted May 12, 2009 at 11:54:19
i do not think voting on articles is needed. this site is an expression of the editors, the comments are the way readers can provide feedback, or by submitting articles themselves. this site is not a discussion forum where everything and everyone carries equal weight. it is a news source / blog. while there is a lot of two-way communication taking place here, it is primarily setup as a one-way communication tool.
By JonC (registered) | Posted May 12, 2009 at 12:38:47
I don't think voting on the articles is necessary either.
What would be interesting is a ratio of total comment scores to number of votes (which would theoretically indicate articles with more interesting comments.
By UrbanRenaissance (registered) | Posted May 12, 2009 at 12:53:04
JonC wrote: "What would be interesting is a ratio of total comment scores to number of votes (which would theoretically indicate articles with more interesting comments."
Not a bad idea, assign the rating to the discussion of the article, rather than the article itself. To that end why not use the ratio of comments that have positive vote counts to the number of comments with negative vote counts. That way you can see at a glance how derailed a thread has become, and at the same time it would (hopefully) prevent people from overusing the down vote since it would make that thread (and their own arguments) appear less relevant.
By jason (registered) | Posted May 12, 2009 at 18:54:18
I can tell how derailed a thread has become when I log in and see that there are 75 comments in 1 day about Burlingtons NHL arena. People simply need to stop feeding the troll.
By hammertime (registered) | Posted March 01, 2011 at 16:18:32
I'm with smz 180, this is a stupid Idea. Why not let everyone think for themselves??
Comment edited by hammertime on 2011-03-01 16:19:49
By hobbit (anonymous) | Posted May 26, 2012 at 15:18:03
comment voting and greying out makes this site utterly childish and monotone. I bet if you did a graph of users you'd a core group of 30-50 people providing 90% of the comment voting. I'd also be willing to bet that 90% of that 90 have their comment settings set so that they read every one. All the greying out does is allow one more way for the herd to tarnish the black sheep. It does not achieve its function of keeping 'conversations' on track. It should be replaced with a function where these 30-50 people who love to be in agreement have the option of sorting the comments so they see the top rated comments first. Then those of us who think those of you who vote on comments are ridiculous wouldn't have to be regularly confirmed of this idea.
You must be logged in to comment.
There are no upcoming events right now.
Why not post one?