Analysis of Scenario Ratings
| Author | Topic: Analysis of Scenario Ratings |
|---|---|
|
Infiltrator
Member # 5576
|
written Monday, February 4 2008 21:05
Profile
Homepage
Inspired by TM's Scenarios-by-Level chart, I've finally gotten around to doing something I'd meant to do a long time ago: making a chart comparing CSR ratings of scenarios. Disclaimer: The categorizations of scenarios, while subjective, are my estimates of community opinions, not my own opinions. As described below, I attempted to organize categories only according to patterns I observed in the data. The chart to which I will be referring is here . When I initially made this chart, it was simply in alphabetical order. However, I noticed immediately several distinct groups of scenarios: those with a small spread in ratings and a high average, a small spread and a medium average, and a small spread with a low average. Then, there were scenarios with high, medium, and low averages whose spreads were much larger. Having seen this, I began to sort the scenarios into the categories in the current chart. After I'd sorted all of the ones that seemed by eye to obviously belong to each group, I wrote a set of rules describing each group. Using made up numbers, (because I don't remember all of the real inputs to the calculation) supposing that the lowest average of a good scenario was 8 and the highest average of a medium scenario was 7, I averaged these and declared that the dividing line was at an average of 7.5. This is the list of actual rules that I derived:
In conclusion: I have no idea what this tells us, but I think it looks pretty. A few final notes: The source code of the program I wrote to draw the chart is available. The data I used was yesterday's data on Aran's CSR summary page . The program is very bare-bones, it reads in a plain text file on each line of which is a scenario name in quotes, then separated by spaces the minimum, average, and maximum ratings the scenario received. I'm hoping to either re-write it to parse Aran's page (or get him to port it to php and have it run directly from his data :P ). I also tested the chart in every web-browser I have; it renders correctly in Safari 3 and Firefox 2, slightly less well in Camino, Opera, etc. Hopefully at worst only the rulers get screwed up, so everyone can see it more or less correctly. -------------------- Überraschung des Dosenöffners! "On guard, you musty sofa!" Posts: 627 | Registered: Monday, March 7 2005 08:00 |
|
Shake Before Using
Member # 75
|
written Monday, February 4 2008 22:18
Profile
You probably shouldn't include the 0-1 bracket if possible, since sub-1 scores are not legal on CSR. Posts: 3234 | Registered: Thursday, October 4 2001 07:00 |
|
Apprentice
Member # 6009
|
written Monday, February 4 2008 22:54
Profile
Pretty neat. Something like standard deviation or variance might be a much more useful indicator of how much controversy is associated with any given scenario's rating... I guess it would take a bit of effort in terms of data collection though. Posts: 18 | Registered: Friday, June 24 2005 07:00 |
|
Law Bringer
Member # 2984
|
written Monday, February 18 2008 00:46
Profile
Homepage
I like this chart a lot. It also makes me wish there was more data - more dimensions as well as more votes... Posts: 8752 | Registered: Wednesday, May 14 2003 07:00 |
|
Law Bringer
Member # 2984
|
written Monday, February 18 2008 00:52
Profile
Homepage
(Sorry, still no edit button.) Also, observation: The masterpieces of TM and Kel are noticably more controversially rated than those masterpieces designed by others. Also controversial (but medium on average) are the works of Jeff Vogel. Posts: 8752 | Registered: Wednesday, May 14 2003 07:00 |
|
Agent
Member # 8030
|
written Monday, February 18 2008 11:23
Profile
Homepage
This is indeed a very nice chart. Great work -------------------- Decca Records - "We don't like their sound. Groups of guitars are on the way out." Posts: 1384 | Registered: Tuesday, February 6 2007 08:00 |
|
Raven v. Writing Desk
Member # 261
|
written Monday, February 18 2008 20:35
Profile
Homepage
ASR isn't controversial though. Great chart. One comment: isn't 'poor' typically a worse gradation than 'bad'? I always remember excellent, good, fair, bad, poor from school (and Angband pseudo-ID). -------------------- Slarty vs. Desk • Desk vs. Slarty • Timeline of Ermarian • G4 Strategy Central "Slartucker is going to have a cow when he hears about this," Synergy said. Posts: 3560 | Registered: Wednesday, November 7 2001 08:00 |
|
...b10010b...
Member # 869
|
written Monday, February 18 2008 21:09
Profile
Homepage
quote:Apparently descriptive linguistics does rot your brain. :P I've always considered "poor" to be less bad than "bad". And the worst Angband pseudo-ID is "terrible"; as far as I'm aware, there's no "poor" pseudo-ID. -------------------- The Empire Always Loses: This Time For Sure! Posts: 9973 | Registered: Saturday, March 30 2002 08:00 |
|
Raven v. Writing Desk
Member # 261
|
written Monday, February 18 2008 22:18
Profile
Homepage
You're right about Angband. I went to look for examples and I found... not much. Most of the rating scales or systems I can find through a quick google search that use bad or poor, do not use both. This is accomplished in various ways -- having only one negative rating, having a more clearly horrendous word for the worse option, "very bad", etc. -------------------- Slarty vs. Desk • Desk vs. Slarty • Timeline of Ermarian • G4 Strategy Central "Slartucker is going to have a cow when he hears about this," Synergy said. Posts: 3560 | Registered: Wednesday, November 7 2001 08:00 |
|
Law Bringer
Member # 2984
|
written Tuesday, February 19 2008 01:17
Profile
Homepage
I haven't seen "bad" in any grading system at school; the worst was always "poor" or "insufficient". I have seen it in a rubric for marking the condition of a library book, though - the scale went "New", "Excellent", "Fair", "Poor", "Bad". Apparently saying that a student is bad would be rude, but the library book doesn't mind. Posts: 8752 | Registered: Wednesday, May 14 2003 07:00 |