Future contest voting changes?

[Fidcal]

Just wondering whether we could change the voting slightly in future contests. Some FMs fall in between so I am agonizing between say 'good' and 'excellent' where 'good' seems unfair but the FM is not 'excellent' throughout. I see we can have 6 choices not just 5 but I have a faint memory that made the calculations more difficult but I can't remember in what way. Anyway, I was thinking something like:

 

• 
   
  
• Poor to Tolerable
  
• Good
  
• Very Good, Excellent in places
  
• Excellent throughout
  
• Near Perfect in places
  
• Near Perfect thoughout.
  

 

Any other thoughts? Or is our present system OK?

 

I hope someone else can organize some of the future contests as my time is better spent on some other stuff.

[nbohr1more]

No matter what descriptors you use, there will always be missions that fall between these ideas. It might be better to simply use a "Star Rating" system with a little more granularity:

 

Example

 

Appearance: 3 stars

Game-play and Performance: 5 stars

Story: 5 stars

 

Contest Element Usage: 1 stars

Technicality: 5 star

 

(edited per Komag's suggestion?)

 

(edit per Fidcal and reality )

 

(So rather than call something poor or near perfect just use star numbers, forum comments can clarify the reasoning... basically the same but the psychology is a little different...)

Edited August 20, 2010 by nbohr1more

[Komag]

I really feel that five levels is perfect. I think you just gotta not be afraid to use a 1 now and then. If you just can't BEAR to hand out a 1 ever, then you effectively only have two levels to pick from. Some people cant bear to even use 2's, so they only have three levels to pick from. If you have a 1-10 scale, only 4-10 ever get used, so it's closer to a true five point scale than it is to a true 10 point scale. I prefer a TRUE five point scale which gets USED. Maybe make 1 sound more tolerable. Leave off calling it "Poor", just call it "Average". Then we can actually use the dang 1's a bit more, and then five points will be enough.

[SneaksieDave]

Well one change I'd suggest is that polls be private (most(?) have been public). That would encourage honesty in voting, and if a poster wishes, they can expound upon their opinions in their post, or leave it at that.

[Fidcal]

nbohr1more: The forum does not permit your idea. It allows up to 3 topics and 6 choices for each topic. So if you mean the player gives a score (choice) of 1 to 5 stars for each topic then only 3 topics are allowed. That is basically what we are doing except each topic is weighted and calculated out of 100.

 

Komag: It's not that I'm afraid to vote 'poor' but there aren't really any that bad so it is hardly used. It seems wasted when I could use an extra place. Plus if we use all 6 choices that gives 2 extra.

 

SneaksieDave: I don't mind either way, public or private. It has no effect on the way I vote. Maybe it needs a poll. I can add one to this thread.

[SneaksieDave]

Maybe it needs a poll. I can add one to this thread.

A private one?

 

<kick, snare-hit, splash!>

[Fidcal]

I'll start a new thread for the poll or it might not be noticed.

[Sotha]

I like the current 1-5 grading system. Adding more steps probably solves nothing. It just brings the problem to a new scale. Keeping the current system also means that older missions gradings still match with the new ones.

 

The problem with the current system probably is, that people feel reluctant to give "poor" or "near perfect" evaluations. It is the choice of words, you see. Someone suggested change of words, but I'm not sure that changes anything. When considering the gaussian curve, AVERAGE is not the lowest possible grade. That way the horrible crap missions would be average. They should be poor and rightly so!

 

My suggestion would be:

a ) stick to the 1-5 scale

b ) remove the words.

 

So the questions would be like this.

Gameplay: On a scale of 1-5, how much did you enjoy the GAMEPLAY aspect of this mission. (5 is maximum, 1 is mininum)

Appearance: On a scale of 1-5, how much did you enjoy the APPEARANCE aspect of this mission. (5 is maximum, 1 is mininum

...and so forth.

 

From the numbers people easily see that missions with grade 1 are poor. 3 are average. And all above that is good/excellent.

 

Note the choice of words: "how much did YOU enjoy the ..." People probably do not have difficulties in intuitively choose a number corresponding to their subjective FM experience. And they do not get a bad feeling of calling other people's hard work "poor." (Even when effectively it does mean that, if they give a 1.)

[SneaksieDave]

The problem with the current system probably is, that people feel reluctant to give "poor" or "near perfect" evaluations.

a ) stick to the 1-5 scale

b ) remove the words.

 

I agree with that. ^

 

Netflix uses a silly description and it hamstrings me all the time.

"Well I didn't dislike it, but I certainly didn't like it..."

 

Everything ends up being voted 3 (of 1-5), which is "I liked it", when I think it should be "average."

[jdude]

I think we should have 1 -poorest 2 3 4 5 - Perfect

 

Just encase somehow someone thinks 1 is perfect

[aidakeeley]

Already brought this up (and was not surprised to have people arguing)...

 

Yes, there should be as much granularity as possible. 1-6 is better than 1-5. 1-100 (a percentage system) would be great. And then of course yes the poll should have a comments section wherein any participant could elect to explain their rating(s).

 

And it should all be pubic because that is the only way, in the end, to really guarantee honesty. The only reason to argue for private is because people want to hide shit. Yeah, that's what it boils down to, on one side or the other, a private this or that is for hiding things.

 

Anyway...

 

I'd give 1-5 a 5.

I'd give 1-6 a 6.

I'd give 1-100 a 100.

 

I'd give "Sucked" / "Didn't suck" a 10...

[LEGION]

Do what thou wilt.

 

I would be happy with any of this solutions.

[Tarhiel]

I´d stick with current 5-level system, or 5-stars system, with proper marking (which means it will be written if "1" means worst or best possible option).

[Briareos H]

I like the 1-5 rating with fixed text (things like "eggsalad" introduce a bias, it's funny but if you want to get serious results you need consistent grade descriptions). I also wouldn't mind a 1-4 rating (poor, below average, above average, excellent), forcing people not to choose the comforting middle ground.

 

Anyway I don't know how you plan to elect the winner for this contest, but to get something meaningful we should open a "pick your winner" thread or at least a mailing scheme. Gathering the aggregated data from individual threads makes no sense as :

1. there is no certainty voters have played all missions (in fact, there is certainty they haven't) ;
2. missions X and Y may be the same kind of "excellent" but different people will still prefer either for whatever subjective reason.

Edited August 21, 2010 by Briareos H

[Fidcal]

I like the 1-5 rating with fixed text (things like "eggsalad" introduce a bias, it's funny but if you want to get serious results you need consistent grade descriptions).

I agree. In the situation it was likely it would not have any effect but its wrong in principle. I think one choice was 'OK' instead of 'good' which is just plain wrong. I'd be reluctant to disqualify anyone on these ground but future entrants should be aware they run the risk of disqualification especially if they push it further. Stick to the same text then we know it's fair. There are plenty of other opportunities for humour.