What's in a score anyway?
Musings on scoring media and why a number by itself isn't insightful
10 Nov 2023I hate giving things a number score. It’s the hardest part of reviewing something because it can feel so imprecise and reductive. It unfortunately also feels like it’s a necessity/expectation. I know I’m guilty about looking for a score immediately whenever I’m reading or watching a review. So given my mixed feelings, I thought I’d write about how I navigate rating stuff and how I avoid relying on just a number.
Setting the scene
Some brief context - since 2017 I have been keeping track of all the albums/EPs/mixtapes I listen to in a big spreadsheet. As part of this, I keep track of the release’s name, artist, genre, and year, as well as a rating. Since about 2021 I’ve been doing the same for the games I play (name, genre, platform, year, rating). And this year I started using Letterboxd for movie reviews and ratings.
Unfortunately I’m an idiot so I have different ways of rating each of these media - I give games a score out of 10, movies are rated between 0-5 stars and music I use what I call The Scale (unrelated to the musical concept of the scale).
Introducing… The Scale
The Scale forgoes any type of numerical score and instead things are just either Bad, Good, or Great.
However by itself, that means nothing. If I said to you “yeah the latest Keshi album was Good”, you might interpret it as I think it’s just an okay listen, that I like it a fair bit etc. It’s got a similar problem to a number score. It’s like how 7/10 to people means different things - some view it as above average, some people view it as the default “average” score like it’s a power-crept 5.
So that’s why I’ve also defined what they mean:
It helps immediately frame what I think of the album. It’s a consistent 5-6 word quick review, which also means I don’t need to write a whole review about it to just say “yeah this is an essential album”. It also helps prevent what I’d call the “score equivalency problem”
The Score Equivalency Problem
A number score in isolation doesn’t reflect the context that the media is being reviewed in, which can lead to things that are not of the same quality being scored the same, and at a glance that looks wrong.
A personal example of this is that on Letterboxd, I’ve given both Avatar 2 and The Super Mario Bros Movie 4/5 stars. I don’t think they are the same quality by any means (I think Avatar 2’s a much superior movie). But these are two movies that are so far from being in the same space that trying to compare them wouldn’t make any sense. Avatar 2 is a great action movie with a length problem, the Mario movie is a children’s movie that I thought had a bit too much licensed music. In the context that I’m watching them, I’m expecting different things and I rate them accordingly. But when you just give it a number and you only see the number, it looks equivalent even though it’s not at all.
The Scale bypasses that by just being super simple and having three things it can be.
Applying The Scale to a number score
So if at its core The Scale exists to provide some framing for what a rating is, why don’t I just apply it to numerical scores? Well good news, I do (at least for video games)!
It’s not a 1:1 copy, but I’ve mapped the numerical scores to a quick guide on what the score means, and it has the same three tier structure.
So while I still give things a number rating, I use The Scale as a guiding light. There have been some games I’ve played which I originally had at something like 8.5, but as time went on and I recommended it to people, I bumped the score up ta 9. Taken by itself, a .5 difference doesn’t seem like a lot, but in the context of The Scale, you now know it’s something I think you absolutely need to play.
Meaningful distribution of ratings
There’s one last piece to the ratings puzzle I haven’t talked about yet. It’s all well and good to say “you absolutely need to listen/play/watch this” but if you say that for everything, it starts to lose its meaning. I’m sure there are some people who do love everything they interact with, but I think the highs and lows need to be a lot less frequent than the middles. To me, it adds the right amount of weight when I do slap that Great (or Bad) rating down. Giving something a high score should feel special.
It’s something I try to be conscious of when I give ratings, and sure enough if you look at my scores for music over the last few years, I’m pretty consistent. Not a lot of Bad, a big amount of Good, and a decent amount of Great.
I don’t think that The Scale is some crazy smart thing I’ve made up - it was originally just an easier way for me to rate things with little to no effort. It didn’t even have a name until I had to come up with something silly to refer it to for this post.
After 7+ years of rating music, I’ve just had a lot of time to think about my methodology and why it feels good to me compared to other systems. Maybe in the future I’ll think it’s too simple, and I’ll just start writing long form reviews for everything I interact with. Or maybe I’ll just stop giving things ratings and instead just remove it from my tracking spreadsheets.
Until then though, I think the The Scale is Great.