Judging: Next Phase
| Author | Topic: Judging: Next Phase |
|---|---|
|
The Establishment
Member # 6
|
written Sunday, December 17 2006 07:43
Profile
All scores are now in, so begins the discussion phase. Before I go ahead and start this, I want to put something forth. I looked through the scores without doing any rigorous analyis and it appears that almost every scenario stands a good shot of advancing to the finals. Now, I don't care if this be the case as if every scenario is very good, all should advance. However, my fear is that there may be a shortage of judges. So, before I post scores, I want to get feedback of the judges and scenario authors. We have a few options: 1) The status quo. Most scenarios will probably advance, we accept this, and whatever judges we can find, we accept their scores. Again, this may lead to few judges making decisions that do not reflect a consensus view. 2) We raise the cutoff to 90% of the top score in the category. This will ensure less scenarios advance, but it would involve changing the rules which is not desirable, but may be helpful. 3) In our discussion phase, the judges agree to try to separate out the scores a little better. Foreseeing this, I proposed that a score of 5 be an average, yet it seems people are still following the old CSR scale. This would require no rule changes, but would require the cooperation of the judges. 4) Some idea I have not thought of that neither significantly changes the rules nor requires judges to refine their scores in a large way. As judges and authors, what are your thoughts on this? UPDATE: Regarding option 2, I did, crunch the numbers (I figure they will be out soon enough anyway) and here are the general resultsfor scenarios advancing versus cutoff assuming I got everything right: 80% -- 10/11 scenarios advance 90% -- 7/11 scenarios advance 95% -- 5/11 scenarios advance I'd lean toward a 95% cutoff or option 3 outlined above. [ Sunday, December 17, 2006 08:54: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Sunday, December 17 2006 09:03
Profile
Homepage
I'd say #2. 3 sounds a little sloppy to me. Of course, I'd like 90%, but that's just because of the selfish reason that I'd have a better chance of advancing. :P Do what you want. Question: is the discussion over who wins preliminary, or who wins final? -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
Off With Their Heads
Member # 4045
|
written Sunday, December 17 2006 09:06
Profile
Homepage
I'm in favor of 2 and 3. Seven scenarios in the finals is about right, regardless of the number of entries. Five would be on the small side, and if they really are pretty near to each other, it might be unfair to eliminate some that are on the brink. Here's the numbers of scenarios in the finals in previous contests: 3rd: 6 4th: 7 5th: 8 6th: 5 7th: 8 With this in mind, 7 seems pretty reasonable. Also, there's no reason we should be abiding by CSR ratings here. These scenarios really should score about the same on CSR, but that makes for crappy contest judging (and I realized this as I was sending in my scores). [ Sunday, December 17, 2006 09:09: Message edited by: Kelandon ] -------------------- Arancaytar: Every time you ask people to compare TM and Kel, you endanger the poor, fluffy kittens. Smoo: Get ready to face the walls! Ephesos: In conclusion, yarr. Kelandon's Pink and Pretty Page!!: the authorized location for all things by me The Archive of all released BoE scenarios ever Posts: 7968 | Registered: Saturday, February 28 2004 08:00 |
|
The Establishment
Member # 6
|
written Sunday, December 17 2006 09:22
Profile
As for the question about who wins what, this relates to the initial round. Of course, this decides who advances to the final round, so it affects who gets to participate in that. I doubt it would change the winner, to be honest, but it would change the number of scenarios judges need to play in the finals. Either the 90% or the 95% does a decent job of cutting things off, nobody, as the scores stand now, is really close and cut. Again, the numbers I put are subject to change based on how judges reallocate their scores. [ Sunday, December 17, 2006 09:23: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Law Bringer
Member # 4153
|
written Sunday, December 17 2006 10:41
Profile
Homepage
I'd prefer #3, actually... I'm beginning to realize just how inflated some of the scores I've given stuff on CSR really are, and that carried over a bit into my scores here. -------------------- Gamble with Gaea, and she eats your dice. I hate undead. I really, really, really, really hate undead. With a passion. Posts: 4130 | Registered: Friday, March 26 2004 08:00 |
|
Law Bringer
Member # 6489
|
written Sunday, December 17 2006 11:33
Profile
Homepage
#2 with 90% sounds good. -------------------- "You're drinking liquor because you're thirsty? How nasty is your freaking water?" —Lazarus Spiderweb Chat Room Avernum RP • Summaries • OoC • Roster Shadow Vale - My site, home of the Spiderweb Chat Database, BoA Scenario Database, & the A1 Quest List, among other things. Posts: 1556 | Registered: Sunday, November 20 2005 08:00 |
|
Infiltrator
Member # 4784
|
written Sunday, December 17 2006 18:13
Profile
Homepage
#2 and 90% I had already decided to be harder on the scenarios in the contest then I would normally do for regular rating. I tried to be tough but fair. -------------------- Forever Always on Past the End TrueSite for Blades - Blades Walkthroughs Pixle Profusion - BoE Graphics Archive Posts: 563 | Registered: Tuesday, July 27 2004 07:00 |
|
The Establishment
Member # 6
|
written Sunday, December 17 2006 18:38
Profile
Unfortunately, others were a bit too forgiving, in my opinion, leading to an overly generous distribution of scores. I've decided to go with the 90% tentatively, and I've also layed out a basic rubric for what score range means what. Hopefully these will be a little more consistent. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
The Establishment
Member # 6
|
written Sunday, December 17 2006 19:12
Profile
Either way, everything should now be set with the private forum. Let me know if there is anything wrong. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Off With Their Heads
Member # 4045
|
written Sunday, December 17 2006 20:17
Profile
Homepage
 This is just a great image. [ Sunday, December 17, 2006 20:20: Message edited by: Kelandon ] -------------------- Arancaytar: Every time you ask people to compare TM and Kel, you endanger the poor, fluffy kittens. Smoo: Get ready to face the walls! Ephesos: In conclusion, yarr. Kelandon's Pink and Pretty Page!!: the authorized location for all things by me The Archive of all released BoE scenarios ever Posts: 7968 | Registered: Saturday, February 28 2004 08:00 |
|
The Establishment
Member # 6
|
written Sunday, December 17 2006 20:22
Profile
I created that account myself. It will be gone before long. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Monday, December 18 2006 12:00
Profile
Homepage
Darn, I thought we finally found out TM's birthday. :P -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
Warrior
Member # 7000
|
written Monday, December 18 2006 12:09
Profile
Homepage
quote:7/7/87 :P -------------------- "It's sort of like Star Wars. Except Jeff didn't make Erika shoot first in Avernum 3, nor did he introduce annoying computer-animated aliens." —Arancaytar The Spiderweb Chat Room Shadow Vale - My site, home of the Spiderweb Chat Database, BoA Scenario Database, & the A1 Quest List, among other things. Posts: 178 | Registered: Thursday, April 6 2006 07:00 |