Score Systems Suck

Original observation, I know. But the fact that I’ve yet to see a professional (or even non-professional) anime review site that dropped this goddamn practice means the problem hasn’t gone anywhere, so I’m going to bang on it even further.

So I was reading Bobduh’s recent Symphogear review last week and my reaction to it was the same as when some Skype friends of mine praise the show – absolute indifference with the occasional facepalm. First off, it stated that Symphogear wasn’t a traditionally good show, which is already a big warning sign because I could say the same for Friday the 13th and despite the negative critical backlash they got, they’re slasher classics for a reason. It then went on to mostly praise plot points rather than what actually happens, which never gives me a good indication of whether the product is worth my time or not (although given how I struggled to marathon S1 and dropped S2 four episodes in, that ship has sailed long ago). As Goddard once said, “it’s not where you take things from, it’s where you take them”, something I apply to practically every medium I interact with let alone anime. So even if I did like Jojo, idol shows, or magical girl anime, saying it’s an amalgamation of them doesn’t tell me anything.

But the one thing I really did not like about the review was the fact that I had to see a score at the end after reading, because I had long since gotten that Bobduh thought the show was fun when he name dropped Chris, and seeing that extra bit of meaningless indication of the show’s quality just made me groan. See, this is one of the big reasons I never bothered to apply for one of these professional anime sites next to the fact that I don’t want to listen to commentaries and they’d never let me namedrop obscure movies and Donald Trump. If I don’t want to see the score, why the hell would I want to write it myself?

Some of you are aware of this, but I’ve never really been in support of scoring things. I wasn’t exactly a dictator about the practice at first and compromised in order to make my opinions appeal more to the mainstream crowd; but my hatred of numbers, letters, “recommended” labels, and such kept growing the more I did it to the point that I got fed up and deleted all the scores on my anime list, leaving people to actually question where they were. They are a poor way to represent complex (and sometimes flexible) opinions regarding what’s essentially unquantifiable art, and I especially hate how MAL requires me to put a goddamn number on every review I submit there – let alone put numbers on all those other categories like enjoyment and character. If it’s good, it’s good. If it’s bad, it’s bad. Why do we have to overcomplicate things regarding whether Accel World is better than Sword Art Online if you don’t recommend either anime to anyone?

There’s also that problem where reviewers use scores as a recommendation to say what their readers should watch without taking their own interests into account. Let’s go back to Bobduh’s Symphogear review again to show you what I mean. He gives the show a B+, which by his standards is a decent recommendation given how often he throws As around. However, if you don’t know Bobduh and hold Bs in high respect like I do, then you’re going to think the show is a high recommendation. And assuming you share my taste, if you didn’t read the review, then you’re going to feel pretty damn betrayed like I was when I jumped into that Now and Then, Here and There anime aka City of God if it wasn’t the least bit good. And I have zero interest in seeing little girls get tortured regardless of reasons, so any general recommendation that includes Gakkou Gurashi in it is going to get broken when I get involved, even if said recommendation also includes Cowboy Bebop and Haibane Renmei.

But the biggest reason I hate scores is because some readers use it as an automatic tool to determine whether or not the review is worth reading rather than the writer’s actual credentials and whatnot. As in, they already have their opinion solidified on the product and want to see it justified. Or alternatively, the number is way different to what’s expected and that’s what draws them to read. Whilst I’m not going to say my reviews are on the same level as Dan Whitehead’s or even Stark700’s, I’m pretty sure a good chunk of the “not helpfuls” (and maybe some of the “helpfuls”) that my stuff on MAL receive are because of the number assigned not being in-line with the common consensus. Then again, the Baccano review I wrote scored alright despite my contradictory opinion regarding how it’s not one of the “best anime EVAR” so maybe my Saekano review really was bad.

No matter how I look at it, scores are a fundamentally flawed system that’s both reductionist and over-complicated. If you want to know whether the review is stating whether this anime is worth your time or not, do the same thing you’d do when judging whether a novel is thought-provoking and well-written. Read the whole thing and decide for yourself. Actually respect the writer by looking at all the words he crafted just for your amusement instead of blowing them off because you noticed he gave a C- to whatever ufotable is producing. Simple as that, assuming the majority of the human race actually has a brain that can make their own decisions based on the evidence in front of them. Which given the number of people who still complain that reviews should be “objective” and that any review that’s based “entirely around the reviewer’s own views” is garbage, wouldn’t surprise me at all if that wasn’t true.

PS: Thank god MAL got rid of that stupid ratio system and now displays top reviews on each anime page based on the number of “helpfuls” only. Now if they’d just make it so I don’t have to score anything.

24 responses to “Score Systems Suck

  1. I certainly agree with you in regards to rating systems like MAL’s, where a lot of very different people just throws numbers and the system preposterously claims to extract objective data from it.

    But I disagree about personal reviews, like a blog’s review. Let me use your own words here: If it’s good, it’s good (so I give it a high rating). If it’s bad, it’s bad (so I give it a low rating). Also, being good or being bad has a lot to do with my own tastes, but I try to ponder some objective things even though it is my own subjective opinion (bad animation is bad, bad storytelling is bad, those are examples of things not subjected to opinion even when I like a show despite them).That’s useful precisely because a lot of people just wants to have his own opinions reassured. I put the rating in the top, not in the bottom, by the way. You know first thing if I find it good or bad, and how good or bad, so I never have to bother writing it and those who don’t like to read different opinions don’t have to bother reading it.

    I just write what I think, what I saw, where I think the show is going (I write episode’s reviews), and that’s what a reader finds in my reviews, agreeing with me or not. The rating keeps me free of some crazy people who bash those who don’t share their opinions (and is fancy).

    • I put the rating in the top, not in the bottom, by the way. You know first thing if I find it good or bad, and how good or bad, so I never have to bother writing it and those who don’t like to read different opinions don’t have to bother reading it.

      But the biggest reason I hate scores is because some readers use it as an automatic tool to determine whether or not the review is worth reading rather than the writer’s actual credentials and whatnot.

      Okay, the subtitles I add to each review sort of gives the game away, but numbers just don’t encapsulate opinions on anime the same way a rating on the new lawnmower model does.

      • I understood when you said that people use ratings to “determine whether or not the review is worth reading”, and it’s true and I like ratings, among other things, exactly because of this. I prefer to not have to engage in futile argument with people who will just bash me because I don’t like the show they like, my arguments be damned. If they see my low rating and skip my review altogether, it’s a win-win situation.

        But of course this is me, you probably want to be read even by moronic haters and I respect that (to be fair I too want to be read, I just don’t want the burden to read and answer their swear-word filled comments). Sorry if my first comment made look like I would want you to start number-rating your articles or that I like number-rated articles more, I were just saying that I like to put ratings in my own reviews and why I like it.

    • I think everything can be subjective, including animation and storytelling, because every judgement is a comparison between what’s on display and some sort of standard you hold. It just so happens that for certain things the standards held by people are less wildly varying, so it’s easier to reach consensus on many things.

      Said this, I do understand the plight for “objectivity”. I basically see it as asking reviewers to try to base their own standards when judging on something they feel is at least an attempt to approximate the standards of their reviews, so that I, the reader, can relate and make use of your opinion. What is called “subjective” is really “niche”, in a way. If you for example make your strong political opinions, or your knowledge of ancient Assyrian literature, a key point of your review, you may have written a beautiful piece but it’ll be completely useless to anyone not holding the same opinions or knowledge of Assyrian literature, simply because their judgement is going to rely on standards so far removed it’ll be completely uncorrelated to the one expressed in the review.

      • Sorry, here:

        ” when judging on something they feel is at least an attempt to approximate the standards of their reviews”

        I obviously meant “of their readers”.

      • Fun fact: I do have something like political opinions and assyrian literature in my reviews. Well, not really assyrian but persian, as I researched it for some of my reviews on Arslan Senki.

        Overall I agree with all you said.

  2. “because I could say the same for Friday the 13th and despite the negative critical backlash they got, they’re slasher classics for a reason”

    No Halloween and Alien are classics of that genre. Even fans of the Friday the 13th franchise know it’s schlock.

    Now the whole rating is never going to go since most people are too lazy to read reviews and like arbitrary scores that don’t tell you anything. Now I do think Roger Ebert did the score system well since he scored films by how good it was for it’s genre.

    “The star rating system is relative, not absolute. When you ask a friend if Hellboy is any good, you’re not asking if it’s any good compared to Mystic River, you’re asking if it’s any good compared to The Punisher. And my answer would be, on a scale of one to four, if Superman (1978) is four, then Hellboy is three and The Punisher is two. In the same way, if American Beauty gets four stars, then [The United States of] Leland clocks in at about two.”

    • Neither of those two horror films you mentioned are slashers.

      I also fail to see how reviews don’t tell you anything unless you’re Devin Faraci. And I’m not big into Ebert’s rating system because genre grouping is a type of grading I don’t care for.

      • While Alien is debatable on the front(the last half is just a well shot slasher IMO),but I fail to see how Halloween considering it’s the prime reason the first Friday the 13th was made(also copied Psycho with it’s score.

        I personally look at reviews for giving me an at what to expect. I don’t make a big deal like most people since it’s an opinion after all.

  3. Scores are good, just not for anime. I don’t see rating scores when I want to have some reference for an anime, I’d rather read reviews from different reviewers and try to find something appealing to me in said show. For movies and TV series though, they work just fine. Take IMDB for instance, everything below 7 or 6 sucks. Now, if the premise of said show interests me, I would still give it a watch and see for myself what’s up with it. Bearing in mind that it would have some major flaws causing it to score badly(either bad acting, bad direction, etc) and may even like it.

      • Your problem is simple: numerically averaged scores reflect average/majority tastes. Your tastes aren’t average, and aren’t aligned with the majority under almost any respect. Hence, scores are your personal scourge :D.

  4. “But the biggest reason I hate scores is because some readers use it as an automatic tool to determine whether or not the review is worth reading rather than the writer’s actual credentials and whatnot.”

    Look, I agree with your post and you have a valid point regarding giving an anime a quantitative rating and this line hit the nail in the coffin. I wrote a Charlotte review in MAL and these rabid fans dismissed mine as a rant from a hater despite me pointing out the goods and the bads just because I gave it a middling score.

    I still give scores to sort these shows to groups but that does not mean (for me) that the shows having the same score are of the same quality.

    My only concern is how should I evaluate these shows should I totally dismiss the scoring system in reviews because no matter how flawed this system is, it is still a good indicator of knowing the great, the good and the bad.

    “If it’s good, it’s good. If it’s bad, it’s bad. Why do we have to overcomplicate things regarding whether Accel World is better than Sword Art Online if you don’t recommend either anime to anyone?”

    Again, I agree but scoring can serve as a warning for those who still haven’t watched those shows and avoid these and maybe tolerate those to some extent. My remedy for this one is to give the readers a thresh hold score for a fine anime (for me, it’s 6/10 or a C+) and a score on which I can say that I genuinely recommend those shows.

    • My only concern is how should I evaluate these shows should I totally dismiss the scoring system in reviews because no matter how flawed this system is, it is still a good indicator of knowing the great, the good and the bad.

      My remedy for this one is to give the readers a thresh hold score for a fine anime and a score on which I can say that I genuinely recommend those shows.

      Or alternatively, you can have the reader read the freaking review.

  5. I agree with your points. The thing I personally hate the most about scores is the reasoning behind them. Reviews should be completely subjective, but scores are supposed to be given according to a certain standard, like in the olympics and shit, and should be rather objective.

    But reviews don’t really use that so called standard, so there really is no meaning behind it. Seeing someone give any anime/game/movie a 1 out of 5, simply because they “got bored” etc always annoys me. Why bother putting a number in the first place if the best you could come up with is “it’s not for me” ?

    Even if your review is very well written, and your scores are supposedly justified, most people just don’t care. Once they see a number that doesn’t line up with what they want to see, they will usually avoid reading it. I much prefer the Steam method of reviews: A like or a dislike. It can’t be simpler than that, and the reviews need to be insightful or funny to be recommended by other users.

  6. “If it’s good, it’s good. If it’s bad, it’s bad. Why do we have to overcomplicate things regarding whether Accel World is better than Sword Art Online if you don’t recommend either anime to anyone?”

    This. Numbers aren’t a very good representation, and whether a show was 3 or 4/10 doesn’t matter. It’s still bad.

    Most anime could fit into 5 categories:

    -Good

    -Bad

    -Dumb entertainment (Milky homes, Beck, Gurren Lagann)

    -Arguably decent show for what it is, but with no entertainment value (Bakemono, Ergo Proxy, Penguindrum)

    -Typical show (shounen/shoujo/slice of life/comedy/etc.) that’s kinda entertaining but doesn’t leave much after a while.

    Almost all shows would fit into one of these five groupings, and those categories would be more revealing than numbers.

  7. I typically take scores with a grain of salt but occasionally find them helpful, especially when I’m familiar with a viewer’s tastes. There are definitely better ways (though certainly less quantifiable and more qualitative) of evaluating an anime. Besides most ratings change over time…I’ve found that a more reliable metric of quality is the rewatch factor–is the show good enough to warrant another rewatch? If not, then it’s probably not as great as you think. Most titles people gush about tend to fade away from memory a year (or even a season later).

    • I’ve found that a more reliable metric of quality is the rewatch factor–is the show good enough to warrant another rewatch? If not, then it’s probably not as great as you think.

      Look forward to next week’s review, where an old favorite gets hammered into tiny pieces.