For Judges
| Author | Topic: For Judges |
|---|---|
|
The Establishment
Member # 6
|
written Friday, December 1 2006 20:41
Profile
Judges have been assigned a specific set of scenarios detailed in this topic. You may judge additional scenarios, but they will be given a lower weight (see below). Submit all judging scores (scale of 1-10, 10 being high, 5 being average, 1 being low) and comments to blades8contest@hotmail.com[/url] by December 16, 2006 at 11:59 pm board time. All scores should be kept private until then. At this date, all comments will be posted on a private forum where a week of deliberations (ending December 23, 2006) will commence. Any judge aside from the author of the scenario may participate in its deliberation topic. Those assigned the scenarios may adjust their scores accordingly, but must give justification for it. If cheating is suspected, the original score will be retained. Those judges who gave extra scores to scenarios not assigned may not adjust their scores. As stated above, any judge may give a review to any other scenario they were not assigned and did not author. Such reviews will count for 40% of a review if the judge does not have a scenario in the same category and 10% if the judge does. At the end of deliberations, scores will be combined and each category will be have a winner declared. Those scenarios with scores 80% or higher of the winning scenario will advance to the final round. Note that if any cheating is suspected (arbitrary scoring, illogical decisions, etc.) scores may be omitted at the discretion of the contest coordinator. Any judge may submit scores for any utility they did not author. Note that some utilities are not platform compatable so adjust scores accordingly. On December 16, 2006, averages will be taken and a winner of the utility category will be declared -- no discussion will take place for utilities. Comments? [ Saturday, December 02, 2006 19:51: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Saturday, December 2 2006 10:59
Profile
Homepage
I'm not sure I like the idea of judges being able to review other scenarios, even with less weighing, because it may still cause imbalance. However, I think that the 'weighted judging' may be a problem, because if you take 60% the assigned judges and 40% the not assigned ones, the non-assigned judges may actually have more impact on the score than they should. It seems like your system goes like this: Judge A (assigned) - 8.5/10 Judge B (assigned) - 8.5/10 Judge C (assigned) - 8.5/10 8.5*.6=5.1 Judge D (not assigned) -7.5/10 7.5*.4=3 5.1+3= 8.1 (total) In this case, the total is less than a straight average, even though non-assigned judges "weigh less" then assigned ones. If you want to keep it possible to judge other than your assigned scenarios, I say that un-assigned judges ratings should be halved or something, with the total possible adjusted, like this: Judge A (assigned) - 8.5/10 Judge B (assigned) - 8.5/10 Judge C (assigned) - 8.5/10 Judge D (not assigned) -7.5/10 (convert 7.5 to 3.8 out of 5) Total possible = 35 Earned = 29.3 8.37 This way, it's higher than a direct average or a weighed system. If this was your plan from start, then please excuse me. EDIT: Did you intend the link to go to the Judging Assignments page? That's not where it leads. [ Saturday, December 02, 2006 11:14: Message edited by: BainIhrno ] -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
Lifecrafter
Member # 6193
|
written Saturday, December 2 2006 11:15
Profile
Homepage
quote:I was under the impression that this was pretty much what *i was saying, but with 40% not 50. Judge A (assigned) - 8.5/10 Judge B (assigned) - 8.5/10 Judge C (assigned) - 8.5/10 Judge D (not assigned) -7.5/10 Total = ((7.5 * .40) + 8.5 + 8.5 + 8.5) / 3.4 = 8.38 If the original system really was that the assigned judges total counted for 60 percent, and the unassigned total counted for 40, then it definitely needs to be changed. But I think that the above was what *i intended. I'm not terribly concerned about how preliminary judging is handled, scenarios are so evenly matched that I'd be surprised if more than 3 scenarios were eliminated. -------------------- Guaranteed to blow your mind. Frostbite: Get It While It's...... Hot? Posts: 900 | Registered: Monday, August 8 2005 07:00 |
|
Infiltrator
Member # 5132
|
written Saturday, December 2 2006 12:14
Profile
Homepage
That sounds good. And with such little time, I doubt many people will get to doing more than their assigned work. -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
The Establishment
Member # 6
|
written Saturday, December 2 2006 12:38
Profile
Lazarus is indeed correct. The case you have would come out to 3.38 using the standard definition of the weighted average (assigned judges have weight 1.0, non-assigned have 0.4, and non-assigned in same category have 0.1). The reason I chose this way was that I wanted more people to judge each scenario, but I didn't want to put too much workload on the judges. This, I thought, was a suitable compromise as it would allow more voices to be heard but not overburden them. I also want focus to be on the actual assigned scenarios, that's why I gave them lower weight. Mainly this applies to if a judge has already played a scenario he/she was not assigned. That judge can review the scenario on the contest scale and submit and still have a voice. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Off With Their Heads
Member # 4045
|
written Saturday, December 2 2006 18:03
Profile
Homepage
This sounds altogether like the experimentation with judging that happened in the Third Contest. I am entirely opposed. I guess it's too late now, though. -------------------- Arancaytar: Every time you ask people to compare TM and Kel, you endanger the poor, fluffy kittens. Smoo: Get ready to face the walls! Ephesos: In conclusion, yarr. Kelandon's Pink and Pretty Page!!: the authorized location for all things by me The Archive of all released BoE scenarios ever Posts: 7968 | Registered: Saturday, February 28 2004 08:00 |
|
Law Bringer
Member # 4153
|
written Saturday, December 2 2006 22:58
Profile
Homepage
Eh, as long as somebody else works out all the math, I'm fine with it. Personally, I plan on just doing the ones I was assigned... and I'd suggest that everyone do the same. -------------------- Gamble with Gaea, and she eats your dice. I hate undead. I really, really, really, really hate undead. With a passion. Posts: 4130 | Registered: Friday, March 26 2004 08:00 |
|
Infiltrator
Member # 5132
|
written Sunday, December 3 2006 15:15
Profile
Homepage
Something just came to mind...having the preliminary discussion forum here might be a problem. TM is judging, and he's banned here. [ Sunday, December 03, 2006 15:15: Message edited by: BainIhrno ] -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
The Establishment
Member # 6
|
written Sunday, December 3 2006 16:00
Profile
For the purpose of this contest, I will allow TM to go to the specific forum for the judging. He will not be allowed to post elsewhere, if he does, he's out. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |