Contest Discussion
Pages
| Author | Topic: Contest Discussion |
|---|---|
|
The Establishment
Member # 6
|
written Sunday, August 27 2006 07:27
Profile
Eligibility Both Blades of Avernum and Blades of Exile scenarios are eligible for entry into the 8th Scenario Design Contest with the following guidelines: 1) The scenario cannot have been an entrant in any previous Spiderweb of Olympia scenario design contest. 2) All entered scenarios (utilities are exempt) must have been submitted to Spiderweb Software.* 3) The deadline for entry is November 30, 2006. * If you already submitted to Spiderweb and either have not been posted or have been removed, you may host it independently. However, you must have actually attempted to submit it to Spiderweb software. Categories Small Scenario Large Scenario Newcomer Best Utility Small Scenario: < 25 zones Large Scenario: >= 25 zones Zones is the total number of towns plus outdoor sections. Near identical towns (such as use of VTE) count as 0.5 of a zone. To be eligible for the Newcomer category, you must not have been a participant in any previous Spiderweb or Olympia scenario design contest. If this applies to you and you have more than one entry in this contest, you may select only one to be in the Newcomer category. Judging Initial Rounds Scenarios in each category will be given a score from 1-10 based on quality, 1 being lowest and 10 being highest. The decision criteria is generally up to the individual judge so long as he/she is consistent where possible. The exception is that a judge cannot favor either BoE scenarios over BoA scenarios or vice versa, each must be judged on it's merits. All judges scores are averaged to make a cumulative score. The highest ranked scenario in each category (except Best Utility) and any scenario with a score of 80% the highest score in the category will advance to the Best Scenario round. For example, suppose a category has four scenarios ranked 8.8, 8.5, 7.6, and 6.3. The cutoff for admission to the final round is 7.04 (0.8*8.8), so the top three scenarios advance and the lowest one does not. Final Round The scenarios that advanced are judged in a final best scenario round. All participating judges then discuss and debate to rank the scenarios from best to worst. Each judge is given three votes that he/she may distribute as he/she sees fit with a maximum of two for one scenario. How the judge actually allocates votes are secret. Once voting has completed the lower 50% rounded down (if there are 7, only the lowest 3 are dropped) are eliminated. If there are ties within the cutoff, all tied scenarios remain. Any votes for eliminated scenarios may be redistributed at the judge's discretion; votes for scenarios still in play may not be reallocated. The process continues until one scenario has the most points. Tie Breaking In the event of a tie that cannot be broken because no votes remain to be reallocated, with the exception of the final decision round, the cumulative score of the initial round is used to make the decision. Should those scores be tied, the high scores are compared and on down to second highest, third highest, and so on. In the event of an actual tie between all scores, both (or all those tied) scenarios are eliminted. If a tie occurs in the final decision round (i.e. two scenarios remain), all votes are cleared and the judges vote for their preferred scenario. If this vote ends in a tie, the above criteria are used. If all criteria are tied, the competition is a tie and both are co-winners. In the event more than two scenarios remain and all remaining scenarios are tied, the original decision criteria is used until two remain (at which a vote is done as above) or the tie is broken. Should the tie be impossible to break, a vote will be done to attempt to break a tie. If doing so is impossible, then all scenarios that are tied are declared co-winners. * * * Comments? [ Tuesday, October 17, 2006 05:06: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
? Man, ? Amazing
Member # 5755
|
written Sunday, August 27 2006 07:35
Profile
Zones, towns, sections. It seemed to folks in the discussion that what mattered for size was the depth of the story. Empty outdoors was small, while a town that changed 6 times over the course of the scenario was large. That is why we talked about adding together non-dialogue scripts to come up with the number of zones. Also, is Best Utility still a category? I remember some discussion on how to best include things like the 3D Editor and Alint. -------------------- quote: Posts: 4114 | Registered: Monday, April 25 2005 07:00 |
|
Triad Mage
Member # 7
|
written Sunday, August 27 2006 08:02
Profile
Homepage
I'd add a best utility category, especially as we would have plenty of entrants. I think that zones is better than scripts - the player still has to walk through 8 empty outdoors sections, which makes the scenario larger in terms of playing (but not designing). -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Sunday, August 27 2006 08:14
Profile
Homepage
quote:I'm sorry, but I'm not sure what you mean by this. [ Sunday, August 27, 2006 08:15: Message edited by: BainIhrno ] -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
The Establishment
Member # 6
|
written Sunday, August 27 2006 08:15
Profile
I would agree with you on scripts as a metric except that a long way back we decided to have both BoE/BoA scenarios be eligible for entry. Since there is no analog for scipts in BoE, it would not be very workable without splitting categories which would make for small categories. Having two types of scenarios allowed will raise interesting issues as far as judging goes. We'll have to iron those out when we see a list of entries. We may have to split the judging based upon BoE/BoA accessibility. Further comments? Btw, does anyone have a list of eligible scenarios? -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
The Establishment
Member # 6
|
written Sunday, August 27 2006 08:17
Profile
quote:Sorry for the double post. What it means is that if you have a scenario, you must submit it to the tables on Spiderweb Software or at least attempt to. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Sunday, August 27 2006 08:19
Profile
Homepage
Elgible Scenarios (all of these have been released) Small: A Perfect Forest/BoA The Final Spire/BoE Redwall: Prologue/BoE A Good Beginning/BoA (?) The Foolish Giant/BoE Nephilim Mystery/BoA Proving Grounds/BoA Large: Magus of Cattalon/BoA Dilecia/BoA (?) Utility 3D Editor Alint HLPM (Kelandon)/BoA Fifth Tower of Magi/BoE Portal/BoE Ultimate Equipment/BoE [ Sunday, August 27, 2006 08:23: Message edited by: BainIhrno ] -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
The Establishment
Member # 6
|
written Sunday, August 27 2006 08:21
Profile
APF would indeed be eligible and will be submitted for those wondering. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Lifecrafter
Member # 6193
|
written Sunday, August 27 2006 08:38
Profile
Homepage
Dilecia has 16 zones, so its a small, although since it'll be in Newcomer I don't suppose it matters. On that note, is a scenario entered only in the Newcomer category? Or is it entered in both Newcomer and the appropriate category for its size? [ Sunday, August 27, 2006 08:41: Message edited by: Lazarus. ] -------------------- Guaranteed to blow your mind. Frostbite: Get It While It's...... Hot? Posts: 900 | Registered: Monday, August 8 2005 07:00 |
|
Triad Mage
Member # 7
|
written Sunday, August 27 2006 08:56
Profile
Homepage
Both categories. Don't forget the graphics shifter for utilities. -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Sunday, August 27 2006 09:05
Profile
Nominally I would say both as well. Of course, we could limit it to one if we want to make things easier on judges. Speaking of judges, accessibility issues creates an interesting challenge. I propose the following: A member may judge a category so long as he/she does not have a scenario in that category. He/she may judge only BoE scenarios, only BoA scenarios, or both depending on the judge's access to software. If the judge does both, he/she must agree to not favor one media over the other -- scenarios must be judged on their individual merits regardless of format. An interesting idea came up that we would also have a BoE and BoA category in addition to the previous ones. This would try to eliminate bias towards one. So basically you would be entered into all eligible categories. Comments? Also, looking at what we have, we may need to reformulate the category definitions. Does anyone know if the bar were lowered to 20 zones if it were to make a difference? [ Sunday, August 27, 2006 09:11: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Triad Mage
Member # 7
|
written Sunday, August 27 2006 09:19
Profile
Homepage
Well, there are three large scenarios in beta-testing: Exodus, E:R, and where the rivers meet. -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Sunday, August 27 2006 09:25
Profile
Fair enough, I think a good approximate ratio to get to is about 2/3 small, 1/3 large. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Infiltrator
Member # 5132
|
written Sunday, August 27 2006 11:56
Profile
Homepage
quote:Yeah, I like that idea too (well, of course. :P ). I was thinking about stating this myself, in fact. -------------------- "Let us see what the new day brings." - Temas, Areni. Visit my realm! Rate My Scenarios! Fort Emerald Robbery The Nephils' Defense The Final Spire The Fifth Tower of Magi The Portal Posts: 626 | Registered: Monday, October 25 2004 07:00 |
|
Triad Mage
Member # 7
|
written Sunday, August 27 2006 12:02
Profile
Homepage
TM and I were talking about three categories: BoE, BoA small, and BoA large. That is, if there are problems getting judges as-is. -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
Infiltrator
Member # 5576
|
written Sunday, August 27 2006 12:17
Profile
Homepage
For utilities: GraphicAdjuster (which I think Drakefyre meant by 'the graphics shifter') and DialogEditor. However, these and other utilites may not fit well with the Submitted to Spiderweb rule. -------------------- Überraschung des Dosenöffners! "On guard, you musty sofa!" Posts: 627 | Registered: Monday, March 7 2005 08:00 |
|
The Establishment
Member # 6
|
written Sunday, August 27 2006 17:04
Profile
Utilities are now exempt from submission as they are another type of beast. However, I think it's only far that Spiderweb get to host the scenarios -- that and it makes judging a bit easier with a central download place. As far as the BoE/BoA categories, I think I'll defer the decision until we have a better idea of the submission and judging situation. [ Sunday, August 27, 2006 17:05: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
The Establishment
Member # 6
|
written Tuesday, August 29 2006 12:35
Profile
*bump* Any more suggestions on the rules? As I said previously, I think the categories are good for now until we see exactly what we have. Until then, any other comments on rules or anything left out? -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Triad Mage
Member # 7
|
written Tuesday, August 29 2006 16:28
Profile
Homepage
I'm not so keen on the point judging system. I wonder what a relative ranking system would give ... someone (maybe me, later) could go through the last contests on the Lyceum and see how the contests would turn out if they were judged in the following ways: MLB MVP voting Instant-runoff system Condorcet method EDIT: A reason for this is to prevent a few people who hate or love a scenario from dragging a scenario to victory or defeat. [ Tuesday, August 29, 2006 16:30: Message edited by: Drakefyre ] -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
E Equals MC What!!!!
Member # 5491
|
written Tuesday, August 29 2006 16:46
Profile
Homepage
As I have mentioned earlier, I much prefer the concept of discussion judging to numeric scores. -------------------- SupaNik: Aran, you're not big enough to threaten Ash. Dammit, even JV had to think twice. Posts: 1861 | Registered: Friday, February 11 2005 08:00 |
|
The Establishment
Member # 6
|
written Tuesday, August 29 2006 17:07
Profile
I don't like Drakey's idea at all. The primary reason is that I feel the past method of doing things based on a point scale appear to work quite well and be the most fair. For this reason, I say we stay with the point system and I'll give an example of why the methods proposed really don't work well here. Suppose there are three scenarios in an initial bracket (say Small Scenario), two are of "about equal" superior quality and one not so great. A judge is forced to rank a top, middle, and bottom. Essentially the scores translated to a 0-10 scale become quantized to 10, 5, and 0. A judge MUST assign one of each, he/she cannot give any measure of relative quality at all other than a pure ranking. Under this system, you can unfairly penalize scenarios of equal quality. At the end of the day, one scenario would advance to the final round and two would not (I'd assume top 67% would be a bit much). Under the current system, the top two would advance if they were of roughly equal quality. Basically, we shouldn't penalize a really good scenario for just so happening to be in a bracket with one of similar quality. Essentially, you will have really good scenarios that will not be able to advance to the final round based essentially on a coin toss of who is judging. Once we get to the final round, such things might be workable as there needs to be a clear winner. But even here, we're really limiting the ability of the judges. Suppose I judge and I really like scenario A and B, but not C. I have to unfairly penalize A or B at the expense of the other rather than just assigning them relatively equal scores. Yeah, either A or B will win in the end, but at least I'm free to rank them near equally. As far as the few people dragging things down argument, I don't see how it helps overly much as judges could conspire and still be unreasonable in their rankings. Also, to the notion of a few people hating a scenario, I say too freakin' bad. We design scenarios for the community. The judges aren't just pie in the sky scenario experts, the judges are part of the intended audience -- the community. If there is a section of people who don't like it, they have as much right to their opinion as anyone else and may give their scores accordingly. Discussion judging -- Interesting and could work. I'm a little leary of it as I feel it's better to have several independent measurements rather than judges discussing and persauding. Maybe this is my bias as a scientist, but I just feel uneasy about it. quote:One more thing, I'd say differently. However, that does not necessarily make the answer more right, just different. I could use the argument had we selected different people to judge as well. Besides, had we used a different system, perhaps people would have judged differently. Basically, any results these tests would give would ultimately be irrelevant as it comes down to opinion anyway. EDIT: Ash's point made me think of something else. I should add that I think judging needs to be done in relative secret until all results are in, at which point the scores and comments would be posted. This keeps the judges as independent as possible. [ Tuesday, August 29, 2006 17:11: Message edited by: *i ] -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Triad Mage
Member # 7
|
written Tuesday, August 29 2006 17:11
Profile
Homepage
I've just done my tests on the 7th contest - it looks like we don't have enough judges for Concordet or Borda - one vote can change things a lot. The results were the same except for Roots and NTH switching (by one point) and Corporeus and Areni switching (also by one point). IRV gives the same results as numerical, but I can see where it has advantages over the numerical system. EDIT: I should add that this was for Best Scenario judging - which happened to be every scenario entered. And I'm not sure I like the discussion judging for something like this - independent judges tend to give more real, unpressured results. [ Tuesday, August 29, 2006 17:13: Message edited by: Drakefyre ] -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Tuesday, August 29 2006 17:13
Profile
quote:For those of us lacking your acuity for these things, could you enlighten? Also, your results are meaningless regardless of what they are for the reason I stated above. They are only different, not better. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |
|
Triad Mage
Member # 7
|
written Tuesday, August 29 2006 17:19
Profile
Homepage
Well, IRV results aren't different at all. The advantage I see is that it's a straight majority vote for best scenario - in a points system everyone can rate a scenario an 8.5 and none of them have it 1st choice, and it will still win. Nobody (except the author) is happy then. After second/third votes are counted for getting a majority for best scenario, it ensures that the best scenario is actually chosen. -------------------- "At times discretion should be thrown aside, and with the foolish we should play the fool." - Menander ==== Drakefyre's Demesne - Happy Happy Joy Joy Encyclopedia Ermariana - Trapped in the Closet ==== You can take my Mac when you pry my cold, dead fingers off the mouse! Posts: 9436 | Registered: Wednesday, September 19 2001 07:00 |
|
The Establishment
Member # 6
|
written Tuesday, August 29 2006 17:25
Profile
I guess we have a different idea of "best scenario" then. I consider "best scenario" to be a cumulative overall thing, not an individual thing for the purposes of these contests. It doesn't have to be anyone's favorite to be "best" it just has to be the most liked overall. I know given the situation of relatively equal choices, I would have a hard time ranking things with as better or worse. I, as a judge, would rather say they're equal and move on. -------------------- Your flower power is no match for my glower power! Posts: 3726 | Registered: Tuesday, September 18 2001 07:00 |