Moo Tang Clan: Measuring the quality of community?

Wednesday, February 9, 2011

Measuring the quality of community?

Wolfshead writes in an excellent post:

Most MMO enthusiasts have noticed a sharp decline in the quality of the WoW community of late. Cursing, nastiness, bullying and other forms of rudeness have now become endemic in Azeroth.
I can imagine at least one prominent blogger arguing that "no, the community hasn't changed, it's just your nostalgic recollection of the past".

Here's a challenge then: how would one measure the quality of community? I'm suggesting quantitative statistics, not gut feel opinions. Do that on a monthly basis, and continue to do that year on year. Get some objective data.


Rebecca said...

Measure incidences swearing on the official forums; only works if posts are allowed if they contain taboo language, but can work if they're posted with censoring.
If they're not allowed, hands-down, you could possibly track how many people attempt to make posts rejected for swearing or other related reasons, but that would probably only be doable by whoever owns those forums.
If you're willing and able to log chat within the game, count incidences of swearing there.

I'm just limiting that to swearing as it's pretty difficult to automatically pick out bullying as opposed to a set of words on a list. Automatic censors do that already; just tell them to count how many times a given swear is blocked and record whatever else data you want, like... faction, date, time, so forth.
Just make sure whatever you do doesn't pick out 'cumulative' or other non-taboo words as swearing.

Number of (official?) complaints relating to bullying, cursing, whatever; admittedly this more measures how touchy the community is getting.
It lets you pick up incidences of bullying and other tougher-to-identify issues; if player-A says 'player-X is harassing me', and the complaint eventually resolves with the decision that, yes, player-X was harassing player-A, then you have a recorded instance of (probable) bullying.
But like rape statistics this only counts the reported incidences. I also don't think most games currently take reports on harassment, scams and other things like this.

Verilazic said...

I would try compiling all forum threads with more posts than some threshold chosen to reduce it to a manageable number, then try to categorize each thread as positive or negative in various ways.