Contest Discussion
Pages
| Author | Topic: Contest Discussion |
|---|---|
|
Triad Mage
Member # 7
|
written Tuesday, August 29 2006 17:58
Profile
Homepage
I guess another part of my disillusionment with the ranking system is that the contest results are basically predetermined based on CSR ratings. This would be a vote for a discussion-based system here, actually. Or at the very least having all scores emailed to the contest runner, so nobody sees other judges' scores until after the deadline to maintain independence. EDIT: Yes I know these are completely different ideas, but they're somehow similar in my mind. [ Tuesday, August 29, 2006 17:59: Message edited by: Drakefyre ] -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
? Man, ? Amazing
Member # 5755
|
written Tuesday, August 29 2006 18:20
Profile
If I do get the honor of judging, I would like there to be be several criteria for scoring a scenario. Has this been discussed already? I'm talking about atmosphere, presentation, scripting "wow"s, etc. Typically when grading a multifaceted thing like a scenario, you can assign letter grades across each category, and then let the judge assign a weighting to that scenario. For instance, if scenario X is really well scripted (A) and is flowing logically (B), but the atmosphere and graphics (E) just make you want to stab your eyeballs, the judge could then heavily weight the atmosphere, award an overall C, and explain in their comments how that judgement was reached. The next judge may not care so much about the graphics, and prefers the story is really cool, weighting that more and having it end up as a B. An arbitrator could compile the weighted scores for each scenario (C and B above) and average them, awarding a numerical winner that is based on player/judging preferences. I prefer to think of this as the Complexosimpleton Methodology of Thought Transferance and Complication. Or CMTTC for simplicities sake. -------------------- WWtNSD? Posts: 4114 | Registered: Monday, April 25 2005 07:00 |
|
The Establishment
Member # 6
|
written Tuesday, August 29 2006 18:40
Profile
The problem with rubrics, as debated many a time by Alcritas and Brett Bixler et al. in the days of antiquity, is that not all scenarios fit the "mold". Building a rubric makes certain assumptions about what a scenario should be like. Unfortunately, it's impossible to make a rubric that is sufficiently detailed enough to be useful while accounting for every creative good idea out there. An example is the issue of outdoors. Some rubrics specifically asked the judge to rate outdoors. This is fine and dandy until you get to a scenario that has no outdoors. Do you penalize an otherwise superior scenario or do you modify the rubric accordingly? Hopefully you would do the latter. Granted, that was an easy one. Unfortunately, in doing so for every possible subtle case you reduce to a final point of just rate the scenario. Of course, you as a judge are free to make whatever criteria you want so long as you are relatively consistent where possible, use 1-10 as the quality scale as described, and do not favor either BoE or BoA scenarios over another. On another topic -- For the record, the ballots should be secret until judging is complete. This is the one way I would conduct the contest differently. [ Tuesday, August 29, 2006 18:42: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
? Man, ? Amazing
Member # 5755
|
written Tuesday, August 29 2006 19:02
Profile
The identity and temporary mailing addresses of the judges should be available prior to judging if there is to be any reliable method of bribery. Just sayin', ya know? -------------------- WWtNSD? Posts: 4114 | Registered: Monday, April 25 2005 07:00 |
|
Triad Mage
Member # 7
|
written Wednesday, August 30 2006 01:27
Profile
Homepage
Salmon's method reminds me a little of staff picks at a book or movie store. -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Wednesday, August 30 2006 03:33
Profile
Homepage
I don't like the rubric idea at all, and I mostly agree with *I on this one. I like the idea of discussion-based judging for Best Scenario. Not for preliminary judging, though, since doing that would probably be more trouble than it's worth. -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
? Man, ? Amazing
Member # 5755
|
written Wednesday, August 30 2006 07:32
Profile
I rate stareye's judging methodology with 5 snowy plovers. (Extra point for fawning support of Bain ;) ) Drakefyre's method is awarded 2.5 rusty nails in an old tin can. By popular choice, as well as sheer momentum, stareye wins the 8th People's Choice Awards. Drakefyre now must straighten out his nails and pound them into the snowy plovers and cook dinner. (Please, someone get me coffee!!!) -------------------- WWtNSD? Posts: 4114 | Registered: Monday, April 25 2005 07:00 |
|
Agent
Member # 3364
|
written Wednesday, August 30 2006 16:20
Profile
Homepage
Consider Redwall: Prologue entered for Newcomer, Small, and BoE if you do it that way. I've sent it to Spiderweb, but they haven't sent me a confirmation e-mail yet. It's available at TrueSite if they don't get to posting it. [ Wednesday, August 30, 2006 16:22: Message edited by: Jewels ] -------------------- "Even the worst Terror from Hell can be transformed to a testimony from Heaven!" - Rev. David Wood 6\23\05 "Do all the good you can, by all the means you can, in all the ways you can, in all the places you can, at all the times you can, to all the people you can, as long as you ever can." - John Wesley Posts: 1001 | Registered: Tuesday, August 19 2003 07:00 |
|
Lifecrafter
Member # 6193
|
written Wednesday, August 30 2006 16:39
Profile
Homepage
I like the idea of judging on a 1-10 scale in the preliminaries, then discussion to decide the final round. The only real problem I have with scoring on a 1-10 scale is that scenarios often seem to receive the same score as they had at the CSR. Which could suggests that judges just take the score they see at the CSR and modify it by half a point or so depending on their personal feelings. Or it could be because the judges are the only people rating it at the CSR, so the scores are just duplicated :P quote:Spiderweb doesn't give confirmation e-mails, at least I never got one. -------------------- Guaranteed to blow your mind. Frostbite: Get It While It's...... Hot? Posts: 900 | Registered: Monday, August 8 2005 07:00 |
|
Triad Mage
Member # 7
|
written Wednesday, August 30 2006 16:47
Profile
Homepage
Right - before CSR existed numerical scores were fine, but now they're just a mirror of CSR. Discussion judging for the finals would be good for me. -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Wednesday, August 30 2006 17:57
Profile
Integrated some of the suggestions. The tie breaking rules are a bit complex, but should be all inclusive and will hopefully not need to be used. Please comment... -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Triad Mage
Member # 7
|
written Wednesday, August 30 2006 18:16
Profile
Homepage
When it says you have three votes to allocate, can I place all three on the same scenario? -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Wednesday, August 30 2006 18:18
Profile
You may, should you strongly favor one over all the others. If this is too abusive, we could max it at two per scenario. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
The Establishment
Member # 6
|
written Wednesday, August 30 2006 19:20
Profile
Sorry again for the double post, but this is a new issue altogether: With the change in rules, it makes it a requirement for those in the final round to have access to both BoA and BoE. I foresee this creating problems. A possible solution is to break the contest into two, one for just BoE and another for BoA. Just out of curiosity, how many people out there have access to both? -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Off With Their Heads
Member # 4045
|
written Wednesday, August 30 2006 21:47
Profile
Homepage
quote:I would be totally in favor of this, if the number of scenarios justified it. I don't think that it will, though. -------------------- Arancaytar: Every time you ask people to compare TM and Kel, you endanger the poor, fluffy kittens. Smoo: Get ready to face the walls! Ephesos: In conclusion, yarr. Kelandon's Pink and Pretty Page!!: the authorized location for all things by me The Archive of all released BoE scenarios ever Posts: 7968 | Registered: Saturday, February 28 2004 08:00 |
|
E Equals MC What!!!!
Member # 5491
|
written Wednesday, August 30 2006 22:01
Profile
Homepage
quote:I do, and will be judging. -------------------- SupaNik: Aran, you're not big enough to threaten Ash. Dammit, even JV had to think twice. Posts: 1861 | Registered: Friday, February 11 2005 08:00 |
|
Loyal Underling
Member # 13
|
written Thursday, August 31 2006 03:07
Profile
Right now I only have access to BoA, and if I get access to BoE (which I'm trying to), then there's a possibility I could only judge newcomer and large. EDIT: Ash, does that mean you'll have no entries in the contest? [ Thursday, August 31, 2006 03:08: Message edited by: the Great Potato ] -------------------- [i]Great Potato[/i] "Unless by the force of eloquence they mean the force of truth; for if such is their meaning, I admit that I am eloquent." -- Socrates Posts: 126 | Registered: Thursday, September 27 2001 07:00 |
|
Agent
Member # 3364
|
written Thursday, August 31 2006 04:42
Profile
Homepage
quote:At least it says they do on the Submission Page. It's not on the Untried table at any rate. quote:Is this new? You can't possibly do this. Staying with the same theme, there are three scenarios in a category, two are basically equal and one is terrible. If the two that are equal result in a real tie, they are both eliminated and the terrible scenario wins. Suggestions for change: Appoint an official tie-breaker. Use CSR's averaged scores. Use CSR's highest score given. Just give them both first(or whatever they tied for) and skip giving a second(next) place. I like the 0-10 ranking for regular categories, but the best to worst for final round. -------------------- "Even the worst Terror from Hell can be transformed to a testimony from Heaven!" - Rev. David Wood 6\23\05 "Do all the good you can, by all the means you can, in all the ways you can, in all the places you can, at all the times you can, to all the people you can, as long as you ever can." - John Wesley Posts: 1001 | Registered: Tuesday, August 19 2003 07:00 |
|
The Establishment
Member # 6
|
written Thursday, August 31 2006 04:53
Profile
You took the sentence out of context. This is after all other metrics have failed. This means that the number of vote points is tied, both cumulative scores of the initial brackets have tied, and all other scores given by the judges are tied. This is a highly unlikely chain of events and will probably never happen. This is what is meant by an "actual tie". Also, it only effects things on the "bottom" where things are to be eliminated. If the top and second place scenarios are tied and there are nine scenarios, only the bottom four are affected. This is where, in this case, there is difficulty in determining who is in fifth or sixth. Finally, the tie breaking is only applicable in the last scenario round. In the inititial rounds, we can have all scenarios advance if necessary so this issue does not come up as there are no eliminations. In the event of a tie in the initial rounds for the top spot, we'll use the highest score criteria and on down. If all match, it is a true tie. Otherwise, it's irrelevant. Again, the tie breaking rules are there to account for unlikely, but potentially annoying situations that could hypothetically come up. I tried to cover every possible case, so they can be convoluted. [ Thursday, August 31, 2006 04:56: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Thursday, August 31 2006 11:08
Profile
Homepage
quote:I'm not Ash, but I can say he won't. We're working on a BoE scenario together, and given the progress rate, I can't see it being released until at least February 2007. Maybe even later. Also, I don't like the idea of breaking it into two contests at all. Even if there are enough entries, that feels really sloppy. Third, I don't fully understand the judging for best scenario. First you say discussion and ranking. Then you say by voting. [ Thursday, August 31, 2006 11:17: Message edited by: BainIhrno ] -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
Triad Mage
Member # 7
|
written Thursday, August 31 2006 12:33
Profile
Homepage
I'm not too keen on having three votes to throw around - maybe you could make each judge pick a 1st and 2nd choice worth 2 and 1 points respectively? Or even 3 and 1? -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Thursday, August 31 2006 13:29
Profile
I don't feel we should force judges to make a first and second choice if they are equal. I don't see any real problems with the three votes other than we could cap it at two for one scenario. It would have the same effect essentially, but a judge is free to pick three choices. Personally, I don't see too much of a problem either way. My purpose is to give the judges a reasonable amount of latitude. quote:What's there to not understand, people discuss/debate, and then they cast a private ballot? [ Thursday, August 31, 2006 13:32: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Thursday, August 31 2006 17:18
Profile
Homepage
But what's the point of having the discussion if the judges just cast a private ballot? I suppose that's what I don't understand. [ Thursday, August 31, 2006 17:18: Message edited by: BainIhrno ] -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
Triad Mage
Member # 7
|
written Thursday, August 31 2006 17:28
Profile
Homepage
So people can voice their opinions but people are not pressured to vote according to the more vocal judges. It helps people consider aspects of the scenarios they hadn't before. -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Thursday, August 31 2006 17:32
Profile
The judges have a chance to debate the others to influence the decisions of others and convince them of favorites and other issues surrounding the scenarios. Judging comments will be made public to judges at that point and they can be challenged/discussed by other judges. The reason for a secret final ballot is to minimize strategizing and being influenced by actual votes. People may vote for things in an attempt to mess with the system. If the ballots are kept secret, none of the other judges know how the others are acting. Just to add, I agree with Drakey here. [ Thursday, August 31, 2006 17:33: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |