23 replies to this topic

[cxwq]

BACKGROUND

There are several major problems with the moderation system
employed by most discussion communities online where
a very small number of people are given a large amount of
power within specific forums on the board

At any given time only one or two mods are likely to
be online. If a problem arises, they are unable to
act outside their own forums.

A renegade mod could nuke an entire forum. Similarly,
a mod with a personal grudge can consistently modify
or delete posts written by specific users.

At all times, mods must exercise personal judgement.
If several mods are not acting in accordance with the
general desires of the community, the site will die.

Moderation is generally all-or-nothing - deciding
what to allow and what to disallow. There is no means
of ranking individual posts for quality.

Mods can not deal effectively with mass spam attacks.


SLASHCODE

Five years ago Rob Malda created a tech-oriented weblog
called Slashdot that had some interesting features
from a moderation standpoint. You can read about their
moderation system here but I'll give you an overview.

Karma: All users have a number between 1 and 100 that roughly translates
to ``how valuable I am to the community right now''.
This number, called Karma, is affected primarily by the quality of your
posts. If people find something you write interesting, funny, or
informative, your Karma goes up. If they think you're trolling or
off-topic, your Karma goes down.

Post Moderation Points: Each post has a score between -1 and 5
that varies as users moderate the post up or down. Each time
someone spends a moderation point on that post, it counts
toward both the post and the Karma of the poster. For example,
if you have some moderation points to spend and you read a
post that is rated at 1 (average) but you think it's funny as hell,
you rate it as such. You lose one moderation point, the post goes
to 2 and the person who posted it gains a Karma point. Now
here's the cool part - each user can set a threshold number on their
profile that means ``I don't want to see any posts rated below this
number''. Never again do you have to read the trolls, the off-topic posts,
or the flamebait.

At any given time on Slashdot there are hundreds of
moderators. Once a moderator uses up their moderation
points, a new moderator is automatically promoted from among
the users with decent Karma. If you are a Slashdot reader with
good Karma, you're likely to become a moderator about once a
month.

NERFHAVEN

For NerfHaven we're planning to take the key components
from Slashdot and modify them to be appropriate for
a discussion board. The main challenge lies in the
fact that Slashdot, as a weblog, has a very high
(about 300:1) post to thread ratio. All threads on
Slashdot are approved and posted by the administrative
team. As a substitute for their high level of thread
control, we're going to set up a thread rating system.
The moderation system will then consist of the following:

User Karma: 1-100, starts at 50. Moderation on your posts
will raise and lower each user's Karma total. Users with
a very high Karma will have the option, on a post by post
basis, of giving their post a +1 rating when they submit it.
Users with a very low Karma will automatically have a -1
modifier on their posts until they return to an acceptable
Karma level. A user who has three posts modded down
to -1 in a 24 hour period will be automatically muted
pending review by an administrator.

Post Moderation: Each post starts (except where noted above)
with a rating of 1. A moderator may spend one moderation point
in order to raise or lower the rating of a post and will be
required to identify the reason from a list including terms
such as ``informative'', ``funny'', ``off-topic'', or
``flamebait''. Users will be able to sort by post rating
or set thresholds for minimum post score they are willing
to read.

Moderation Points: At any given time there will be a large
number (dozens probably) of moderators. Each moderator will
receive 5 moderation points to spend in each 24 hour period.
Any unused moderation points disappear after 24 hours and the
total is reset to 5 for the next day. Every few months a set of
new moderators will be chosen at random from all users
with at least an ``average'' Karma and at least a minimum
amount of time since account creation. Moderators may be
demoted and replaced for any of three reasons: consistently
not using their moderation points, a track record of inconsistent
or incorrect moderation, or if their Karma drops to a ``below average''
level.

Thread Ranking: All users will be able to rate each thread
on a 1 to 10 scale. The average of these scores will constitute
the Thread Ranking. Users will be able to (optionally) sort
threads by rank rather than date or set a threshold such that
they no longer see threads ranked below a specified number.


Thoughts?

[Groove]

Hrm. I don't know if I like the idea that Mods are chosen at random, that kinda buggles me. Sure, it all has to do with Karma, but, there's bound to be a sour one amongst the crowd every now and then that probably doesn't deserve his Modship. I do like the Karma system, though. It sounds logical, fun, and appealing. Is there another way to chose Mods?

[VACC]

Well Groove you have to understand that first of all, mods are chosen from a restricted pool, based on their karma and past records. Yet, that is really beside the point. You see, the whole idea behind this system is that no one is given a great amount of power. Each mod only has enough power to make a small amount of change, and only combined, as a community, can the moderators make great changes. If one acts up the others will fix his mistakes. Of course there will be ways to remove moderators prematurely if they do show signs of abuse.

VACC

[Groove]

Yeah...that was my next question. If a Mod goes postal and abuses his power, there would be ways to remove him. Kinda makes sense now...Oh, and VACC, by the way, I've re-drawn you in this new style I'm likin' greatly. I'll get it to you this weekend along with this new comic, C.

[cxwq]

Exactly what VACC said.

The more you spread out the power, the more reliable it becomes. One mod going postal is a big problem when you have half a dozen mods. It's a very small problem when you have the same power spread between 30 mods. Nothing will really happen on the boards unless at least two or three mods feel the same way. If the mods are evenly split on whether a post is good or bad then nothing will change because they'll cancel out.

The randomness in selecting mods is also a good thing because the site will reflect the average view of the community better. And since anyone could become a mod the next time around, it's an incentive to post useful stuff and keep your Karma high. That way VACC and I won't be busy playing favorites and having people feel like they need to kiss our ass to be an active part of the site.

Good thoughts! Anything else?

[Sandman]

First off, thanks for inviting me to preview your site!

I talked to VACC a little bit before this, so I'll transplant a view of my thoughts into here. First off, I'm under the impression that NerfHaven is going to be mostly forum-focused, but I'd still like to see a little more content. While I am really eager to see the revival of Groove's comics, and it's always nice to see well documented modifications, I still think that it would be nice to see more modifications, homemade projects, and possibly some biased reviews!

As for the forums, it really shows through your thoughts that you want them to be the best set of forums Nerfers have ever seen. The planned moderation system sounds like it's going to kick some serious butt, but I have a few things to bring up. The karma cycle is a very ingenious way to moderate forums, but I'm skeptical as to whether it will prove to be effective without a large (and I mean LARGE) base of users that post consistantly. This opinion may be so just because I've only seen what I have seen, and this setup is going to be something extremely different. My concern has been that other Nerf forums don't seem to be active enough to truly be able to take advantage of something like "Karma". There also doesn't seem to be too many outbursts on the forums, and there hasn't ever been (to my knowledge or memory) an admin or moderator that has gone postal on the forums (someone tell/remind me if this HAS happened).

This leads me to my next concern. How are you guys planning on getting the word out on the site? You can only get so many people to transfer over from the other forums (most notably NC and NHQ), and while those people can invite all their friends, there's no telling how long they'll stay or how much they'll post. I think back to NerfOnline before it was melded with X-forums, when there seemed to be a continual flow of new members. Why was this so? I think it's because when you type "Nerf" into just about any major search engine, NO was on the top (or near it) of the list. Any plans on getting some publication somehow?

Anyway, best of luck with programming this Karma system into the forums. Let me know if you need some help with the forums or anything. I've got projects of my own to work on, but I'd like to help if you need it or if I can.

[cxwq]

Thanks for stopping by to take a look!

We would love to see more content here and are going to work very hard to make sure that there's plenty to look at beyond the forums. We're going to need lots of help on this so if you have any ideas feel free to wing them our way!

You bring up an interesting point regarding the Karma system and the size of the site. Other sites with similar systems are all large. They went to a system like this out of need. I want this system because I see it as a kind of democratic ideal. As to how well it will scale to a smaller site, time will tell. What works for Slashdot with several thousand posts per day may not be cut-n-paste effective for us. There may need to be a lot of tweaking in order to make things work the way we want.

As for search results, meta tags and external linkage make the world go 'round. Try Google searches for:

nerf blastfire mod
nerf eab mod
[url="http://"http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=nerf+sm1500+mod"]nerf sm1500 mod[/url]

I pulled that site down two weeks ago and Google still thinks it's the shit. When we go public, NerfHaven will be the king of search results.

Welcome to NerfHaven!

[merlinski]

I'd also like to thank you for letting me preview this site.

As for the mod system, I think once you get a large number of members, It'll work incredibly well. The only problem might be when the site starts gaining size, but isn't very big yet. I do, however, think that by inviting certain people who you trust to preview it, you avoid that problem. I'm assuming that the first time you choose moderators in the way you described, almost all will be the people who were here before it "went public". I think that will insure that the system works. I really like the idea, a lot more than the moderator system used on other sites (only having limited power and being the only mod to check the boards can really bug me). I also have no doubt that you'll find the member base you need.

As for content, I think that you don't need to worry about that too much, considering that the most successful part of any nerf site has always been the forums. I think that it would be possible though to get people to write reviews, mods, stuff like that. I know I would be more than willing to help out with any of that, and I think there are quite a few other people who would too.

One more thing, if you ever want some help with the actual structure or appearance of the site, I may be able to help with that too. I mostly do flash stuff now, and I'm getting pretty good (actually getting paid for it too ).

[Zero Talent]

A unique system, indeed. I personally don't trust the outcome of a democratic system with the way the maturity level is turning over at the other boards, but it's not my say; The community is itself, not a select oligarchy.

Sounds excellent in theory, though on the idea of moderator points expiring after a period of inactivity; What if they don't need to be used? If this is a semi-permanent loss of moderator ability, the community could be crippled after a small period of peace and pacifism. If it's just temporary, then yeah, sounds good. Otherwise, it looks to be theoretically stable, even with a low population.

Ah, and while I'm here, thanks for the e-mail.

[cxwq]

Ah, but you don't use your mod points only for bad stuff!

Ideally, most mod points will be used for increasing the value of a post. That way you can see at a glance which posts might be really interesting as well as which ones are definitely to be avoided.

[Paradox]

Interesting... I gotta few questions though... Please don't be offended if I'm playing devil's advocate in this.

1. Often as not people attacking the board don't care if they are still around tommarow. Is there a way to deal with such people quickly?
2. I'm still of the opinion that allot of 'mod points' will probably go to decreasing a value rather than increasing it. I mean, if a mod comes upon a topic thats gone off-topic and has several off-topic replies, aren't most of the mod points going to go into that? Or if someone flamebaits or flames and gets replied to with additional flamebaits/flames...? Those MPs could go pretty darn quick (if this board becomes more active). And if someone is doing something REALLY inappropiate on several boards the mods will basically have to gank the person with negative MPs, which in turn could temporairly cripple the system. Also if there are organized trollings (heck, didn't we have some of those on the old NO?) then the mods and mod points could be constantly diverted and drained for a period of days with too many posters to organize a response to.
3. Not to mention there could be HUGE potential for large, popular movements abusing the system to kill people that disagree with them, even on Nerf matters. When I joined up to NO when it was nerfonline.com (the one with the white background if you don't know what I'm talking about) a guy got mass flamed for saying the FireStorm (a gun a surprising number of people like now) was his favorite gun. Theoretically, this person could've been unfairly bumped vveeerrrryyyy low on the karma if the flamers had had this power. Also, to be frank, I'm not too sure of my OWN security under such a system given that the Nerfonline community-at-large tends to be leftist and atheist, when I'm a Catholic Right-wing conservative... Even though my arguements have almost always been very appropiate (maybe Cwxq remembers). In short, there is potential for people to be injured because the content of their post, even if valid, is against the general community's opinion (ergo also the majority of the mod pool), which in turn opens that clique to retalition from people they may've bumped down but not out of the mod pool. So much potential for what were personal conflicts to have effects on the board. Heh, wouldn't the Lozi's have loved this?

Admittadely, these are fairly extreme cases. But most of them have basis in the community's history.

[Supreme Admiral FDG]

I think that this whole karma/rotating mod system sound interesting. I have my doubts about it, but I will wait and see how it goes, I wish you the best of luck with getting the system to work properly.

As to the rest of the site, I like it, good job.

[VACC]

Interesting... I gotta few questions though... Please don't be offended if I'm playing devil's advocate in this.

Well I think a lot of the issues you bring up are ones that we will have to take into account as we grow and develop. This is precisely why we've known from the start that despite our best attempts at autonomy, our job developing and tweaking this system will likely never be done. However, I think I can answer a few of your misgivings right here and now.

The moderation pool is what we're calling all applicable candidates to be moderators. This pool will be created through the application of criteria to the general public in order to dismiss those that would be either more likely to abuse power, or those that simply have already shown that tendancy. We also will maintain a list of those who have abused or misused moderation privledges in the past, and we have retained the option to remove them as soon as they become evident.

However, I think the main obstacle to these problems is what we're hoping will be the new attitude of our members. We have created this site in the hopes that it will create some kind of harmonious environment for nerfers. The ability to moderate posts up or down in level, and the ability to view only selected levels of posts will, we hope, keep people relatively happy with their chosen environments. If you don't like this or that, you basically don't have to put up with it, and therefore should not grow any kind of animosity towards the other board members. Now we know that's too perfect to be true for everyone, but that is why we will use the moderator pool to remove the ability of people we see gaining specific and overt dislike for others from eligability.

VACC

[merlinski]

Paradox, you have good points. However, I think that if the system is successful, there won't be a need to depend on untrustworthy people for moderation. As to your concern about bias making disagreements dangerous, I feel that for the most part, there haven't been too many examples (recently) of people being persecuted for disagreeing with the general consensus. And to me it seemed that the majority of nerfers are more conservative than liberal (I've usually been one of a few on the liberal side of most arguments). Maybe that has something to do with the general mindset of Conservatives... hey, look at the NRA

[cxwq]

Thanks for the thoughts, Paradox - I'll address a couple of them.

First, regarding mass troll attacks and single lusers using up all the mod points on their lameness, there are a couple things in the system that will help with this. Firstly, It only takes two mod points to take a post from 1 to -1 and make it invisible except to mods. Further, if this happens to three of your posts in a 24-hour period, you're muted pending admin review. So it takes precisely 6 mod points spent by at least two mods to mute someone. The second thing is that there will be at least email validation on new accounts, and possibly admin auth. We're frankly just not that interested in having the people who need an account now and aren't willing to wait for auth. That should reduce the potential for attacks significantly. When was the last time you saw mass trolling on NC?

Regarding abuses of the mod system, there will be very clearly stated rules for moderators. A consistent track record of abusing mod powers will result in permanent removal from the potential mod pool. We may automate this at some point in the future by adding meta moderation where users can rate the fairness of each mod and that rating goes back to the karma of the moderator <vbg> but that's a ways off, if at all. In any event, VACC and I will be happy to unmod anybody who uses their points in violation of the (yet to come) moderation FAQ.

Thanks again for the thoughts!

[Death]

Forgive me if this has been addressed... Only posting during a quick break from studying. If we have switched the focus of karma from individual posters to threads, then how will the moderator pool be chosen?

[cxwq]

Users have Karma which is based mostly on moderation done to their posts. Secondarily, at some point, it might be based somewhat on the subjective quality of their moderation, given that they're a moderator.

Threads have a ranking that's just the aggregate of how all readers (not just moderators) have rated it.

[VACC]

Sometime we measure our dicks and rank things accordingly as well. Just some freiendly advise, don't post on cold days...

VACC

[GunnedDown]

I have talked with Groove over AIM about the site and this system, and I noticed another possible flaw. Groove told me about the ability to "block" people below a certain karma, which you could set. Now, say, most people would probably choose to block out people of a general area of a karma. Not everyone would, but most people would, or so I would think. But if a user has a karma that falls below most of the people's standards, they won't have an efficient way of getting back up, even if they start to post some nice posts and realized how the system works. Although maybe a few people could read the posts of the low karma users, and then decide to rate their posts better, it wouldn't be sufficient enough to push them back to view of the general public, without the person posting an abnormal amount of posts. Maybe I missed something, but it seems like this will occur at some point.

[cxwq]

Moderators won't be able to filter their view by post rating. That's the one down-side to being a moderator <g> but it insures that every post gets evaluated.

Thanks for the feedback!

[GunnedDown]

Ah, yes. I see now. That would make much sense, especially since a sizeable percent of the current users would be a moderator. It will work fine then.

[Doogie]

The idea seems rock-solid, and I approve.

[Oddball]

EDIT: Argh, nevermind, its on the other thread

Edited by Oddball, 14 January 2005 - 10:28 PM.

[xedice]

found my answer,

sorry, old topic

Edited by xedice, 18 January 2005 - 01:12 PM.