Voting Conventions

[Springheel]

With nearly every mission getting a poll now, how are people handling their voting choices? Are we supposed to be voting for missions on some objective standard compared against all other missions?

 

For example, when voting for the Halloween missions, are we supposed to be taking the speed-build component into consideration? If I vote 5 for aesthetics for a speed-build contest, does that mean it's great aesthetics give the amount of time allowed, or does that mean it ranks next to Alberic's Curse?

 

I realized when voting for The Creeps that I was kind of comparing it to non-contest missions when deciding what rating to give it. If I had been only comparing it to the other Halloween missions I would have ranked it higher.

[nbohr1more]

The implication has always been that all voters should compare the missions against a universal ideal of what the perfect mission should be like.

 

In practice players are biased from both the immediate competitors and the history of all FM's.

 

The best we can hope for is that the voting be consistent between the competitors.

 

If you vote based on a universal ideal for one contest mission you should hold the other contest missions to that same standard.

 

If we are reasonably confident that the players are doing this, we can then go back and poll them regarding "all time rankings" and

find out if the standings aren't correct then calibrate all the results for each contest by an offset value.

[stumpy]

I got asked to add a poll to lord duffords so it could be voted on.

[Sotha]

The only way to keep this sane, in my opinion (with which you are free to disagree), is to encourage people to vote each mission as per overall all missions and not taking into account any speedbuilding or other factors.

 

That way all the missions rankings would be somewhat comparable.

 

But what I saw in the contest FM discussion threads, people gave gradings according to external factors and not the mission itself.

 

Therefore the comparison to other missions is already broken. Maybe we should make some sort of voting standard. Standard poll with voting instructions how to rate. Then mash these instructions always into contest entry threads.

 

Alternatively, we just don't mind the ratings. The problem is, that some new people might use the ratings to choose which missions to play. Then they might be stunned why a mission with really good ratings isn't that great.

 

There was discussion of making an ingame mission rating system. After completing a mission, you would get to rate it and the ratings were sent to some TDM mission database. Maybe the mission downloader would show the average rating the mission has. But I'm not sure if this is a good idea. Sure, it is a nice tool to check which missions to bother to play, but it is bound to be abused. And people will get upset if their mission gets mangled with ratings.

 

Or maybe we build a yearly mission ranking list? At the end of each year, we would hold a big community poll where everyone gets to vote their favourite missions. Then we build a top 50 TDM mission list. And maybe put it in the wiki. It might serve newcomers to find the best missions. The problem is, that the forum software won't probably support such a big poll.

[nbohr1more]

Your last suggestion seems to be the best. An annual "top missions" vote. We just need an interface for it...

[RPGista]

As a newcomer I always saw polls as merely a tool for map authors to get a concrete feel of the success/failures of his/her own missions, a kind of "numeric review", if you will. Whenever I play a mission, I try to understand what the author really wanted to achieve, the intentions, the overall feel, the execution, and thats my only criteria to vote - how well he achieved what he wanted. So missions like Heart of Salvation, Score to Settle or Thiefs Den were all pretty much perfect in their own different scales and scope. Creeps is brilliantly executed, but has slightly simplistic architecture on parts of the map - by voting you can tell the author what he got right and what wasn't as great, and thats all there's to it, for me. I made the mistake of giving it a sufficient score on visuals because I was giving missions that were a lot more complex (architecture wise) "good, but not perfect" ratings. After thinking for a second, I realised that the atmosphere of that map alone easily surprass that kind of nitpickng, so thats a perfect example of why missions shouldnt necessarily be compared to others.

 

Missions with high ratings would basicly mean their authors achieved exactly the gaming experience they were going for, and not necessarily that one mission is "better" than the other. Crude missions could very well be kept from the suggested or the downloader list (freely available in the community, of course), but to tell you the truth, I plan on playing ALL missions available and havent checked ratings or reviews for any of them before playing (no spoilers), Im now going in chronological order, actually.

Edited November 24, 2011 by RPGista

[Glyph Seeker]

I deliberately don't vote on missions that I didn't enjoy - I feel horrible denigrating someone else's work since I do not map myself and can't give really useful advice about building a map better.

[grayman]

My vote would be to only vote on contest missions. Then that vote would be relevant for that contest, and have nothing to do with other contests or missions that weren't done for contests. If I'm a newcomer to TDM, and I haven't played many missions, but I'm playing a set of contest missions, why should I have to worry about how they stack up against other missions I haven't played? If I'm expected to do that, then you're asking me to play everything available before voting. What do you do when someone releases a new mission 3 months later? Go through in some "official" manner and re-rank everything all over again?

 

I don't listen to book or movie critics; I make up my own mind. Some folks like horror, some like mansion heists, some like stories with depth, some like to ghost. And the trolls like to drag everything down.

 

So I would say don't make an overall ranking list and get all hung up in the math of the thing, trying to keep it up-to-date, worrying about whether enough people voted to make the rank mean something, worrying about disregarding a certain person's crappy vote, etc. etc. etc.

 

Keep it simple. Just vote on contest missions.

[jaxa]

Voting will always be subjective and arbitrary. Add your polls with that in mind and take the results with a grain of salt. There are no scores on the mission download page or the in-game downloader so I doubt anyone is getting put off of individual missions. Polls are for mappers to gauge opinion and to determine the results of friendly contests.

[AluminumHaste]

The implication has always been that all voters should compare the missions against a universal ideal of what the perfect mission should be like.

 

 

I completely disagree. That should only be the case with regular missions. Anytime a mission is created under constraints or limits (e.g. contests, or speedbuilds) those missions should only be compared to those other missions in the same class/category.

It is completely moronic to compare Return to the City V2 versus let's say any of the Halloween contest entries for obvious reasons I am not going to state.

 

Just my opinion.

[nbohr1more]

That all depends on what you want the voting results to show.

 

If you want all the missions to be ranked so that newcomers can see the best verses the worst then you vote against the ideal.

 

Yes, if you are only subjectively rating missions against other contest entries then it's wrong to hold missions to the standard of best of the best.

 

I agree that the existing vote result and methodology is sloppy for the purpose of "ratings for new players".

 

And since it's a public vote, the reliability of the "judges" is quite dubious ... (Fortunately vast numbers of votes tends to average out outliers )

 

Yes... there are far too many psychological flaws in asking voters to come-up with an absolute ranking...

 

Again, a direct vote for "best all-time missions" instead of relying on contest votes would help solve that issue.

 

Whether there is a need for such a ranking? Not really. These missions are free and generally of good quality.

 

Maybe someone out there want's a "do not play list" but that wouldn't be a very nice thing to hold a vote for...

Edited December 5, 2011 by nbohr1more

[AluminumHaste]

That all depends on what you want the voting results to show.

 

If you want all the missions to be ranked so that newcomers can see the best verses the worst then you vote against the ideal.

 

Yes, if you are only subjectively rating missions against other contest entries then it's wrong to hold missions to the standard of best of the best.

 

I agree that the existing vote result and methodology is sloppy for the purpose of "ratings for new players".

 

And since it's a public vote, the reliability of the "judges" is quite dubious ... (Fortunately vast numbers of votes tends to average out outliers )

 

Yes... there are far too many psychological flaws in asking voters to come-up with an absolute ranking...

 

Again, a direct vote for "best all-time missions" instead of relying on contest votes would help solve that issue.

 

Whether there is a need for such a ranking? Not really. These missions are free and generally of good quality.

 

Maybe someone out there want's a "do not play list" but that wouldn't be a very nice thing to hold a vote for...

 

Good points all around, didn't think of that. Thanks

[nbohr1more]

Take that with a grain of salt. As a promoter I can make recommendations to players that may not reflect the reality of what players prefer. There is no objective list of rankings therefore my word is gospel and I am God. I don't think I've abused this power too much but... Should there be a listing just to keep me in check?

 

LoL

Edited December 5, 2011 by nbohr1more

[Springheel]

Anytime a mission is created under constraints or limits (e.g. contests, or speedbuilds) those missions should only be compared to those other missions in the same class/category.

It is completely moronic to compare Return to the City V2 versus let's say any of the Halloween contest entries for obvious reasons I am not going to state.

 

Yes, I think this is actually the way to go. Contest missions get ranked against other contest missions, non-contest missions get ranked against other non-contest missions.

 

Again, a direct vote for "best all-time missions" instead of relying on contest votes would help solve that issue.

 

I think this might be a good idea as well. We seem to regularly get threads where people ask for mission recommendations...it might be worthwhile having a vote every year for "Best Mission", "Best Mission for Newcomers", etc and posting that list somewhere.