Reputation and Ranking Systems – Identifying useful people and content

This session had about 50 people with no dominating presenters - rather short comments from perhaps 20 people. Jim Cashel moderated.

Three roles of rating

• signal information
• changes behavior – incentive effect
• attracts participation of high quality people

Reasons not to rate

• Ranking makes some veterans seem like expert and so prevents new people from participating
• It’s a disincentive for people who post rarely.
• Reputation is more of a popularity statement rather than quality reflection.
• Schwab Foundation – parents of kids with learning disabilities – doesn’t like reputation systems, since their kids get ranked low all the time. They only show how long has someone been on the site. they He have a group of core users who don’t want to be identified as core users – so they are not displaying how many total posts.
• On the Well, many people hate each other politically – they had to create anonymous judging and people hated it because they voted for people they didn’t like.

Techsoup's levels of recognition

Techsoup has come up with several stages where helpful members are reconized. If they post 200 time and if the Community Manager thinks they are good posts, then they get a star. The next step is that they get to be forum host, after that they can host an event and get more admin privileges like access to a private forum. At some point they start to receive $150 a month. Techsoup wants to figure out how to make it possible for the community to decide on who gets these priveledges.

Use of alias vs real name –

• Monster needs to be anonymous since people are looking for jobs.
• It’s also needed where people are oppressed.
• There’s a different model for people under 25 who don’t care about being anonymous – they use real names on places like MySpace.
• The Well wants to have people use their real name and others ferret out the people who don’t. The people who don’t use real names aren’t taken seriously, get accused of misleading others. It’s hard to mix the two.

Misc –

• A "That’s so you" button allows members can use it to show what others think is a typical post of a particular person then it shows up on their profile.
• Having a face-to-face component increases trust.
• It’s less disturbing to give negative comments on the content than on a user.
• Reputation is just how you are ranked but who you associate with – like Linked-In or buddy list.
• Consider giving more weight to the rating of those with higher ratings.

Please add your questions or comments and watch this space for the next two sessions.