What's in a Number? THE LAST OF US and the Folly of Metacritic

RantGames by Christian Mills

The Last of Us is a great game, a wonderful game even, with an incredibly immersive storytelling, beautiful sound, and tense combat. But is it perfect? No, far from it. Graphical glitches are abundant and even occasionally game breaking.

On the first mission, you are running away from infected zombie-things (no, it’s not a spoiler since it happens in the first 5 minutes of the game), and if you run too slowly or take a wrong turn, your character dies, and you need to try it again. This is a tried and true formula for chase scenes in games, and it works fine -- until the environment doesn't load fast enough, so you die by running into an object you cannot see. And that happened while I was playing. Twice. I turned the corner, desperately trying to escape death and was faced with the beige void of missing textures. I stumbled through the blank environment in a feeble attempt to continue unmolested, but it was not to be. Once I respawned and reached the same point, I turned the corner and beheld, instead of a beige nothingness, a patio crowded with tables and chairs. My cause of death now became obvious; there was no way I could have made it through all of that furniture running blind. This simple bug by no means ruined the game for me--in fact, it just gave my wife and me a good laugh as I continued on. What it does mean is that the game is far from flawless, so why did it get so many perfect scores?

To be fair, not all rating systems claim a 10 score as “perfect”. IGN's rating scale judges a game with a 10 score to be:

“The pinnacle of gaming, a masterpiece may not be flawless, but it is so exceptional that it is hard to imagine a game being better. At the time of its release, this game is the not just the best the system can offer, but better than we could have expected.”

IGN recognized that the game doesn't need to be flawless, but “hard to imagine a game being better?” It is actually very easy for me to imagine how to make The Last of Us better. How about no graphical glitches that cause my character to die? Or visible seams in dark tunnels, where you can peek into the glowing, white void beyond? Or a glitch where your health bar always looks empty, so you just have to guess how much damage you have taken? All of these are little frustrations that only slightly lessened the experience for me. All-in-all I would give it a 9. It's an absolutely wonderful game with just a few weird spots here and there. But with so many obvious flaws, how did it get so many 10s?

One major factor is game reviewers’ tendency to rate games at extreme ends of the scale. If a game reviewer enjoyed a game, they want to give it a positive score, so above a 5, but most people still regard a 6 to be a sub-par game. This causes many reviewers to give decent games scores of 6.5 and above, when honestly anything above a 5 should mean “not bad”. So you end up with a scale starting with okay games at 6.5-7, good games at 8, and great games at 9. But what do you do when you run into an amazing game? A game that will define a console generation, but is flawed? “Is it really only half of a point better than that 9 game I played last week?”  And no, it’s not. It is at least 1 point better, but that lands you on a 10. So that’s the score the game receives, thus perpetuating the cycle.

If that is where the effects of an inflated rating system ended, there wouldn't be a problem. Imperfect games getting 10s is really not that big of a deal, but the consequences are much farther reaching than that. A disturbing new fad has developed where game publishers reward or punishing game developers based on a game’s Metacritic score. One prime example is Obsidian, who created Fallout: New Vegas. According to a tweet from Chris Avellone, a designer for Oblivion, they would have received a bonus if the game scored above 85 on Metacritic. It only received an 84, so Obsidian had to go home without a healthy bonus in their collective pockets for a game that received 36 positive reviews and only 3 “Mixed” reviews and no negative ones. The lowest reviews it received were two 65s and one 70. Those three reviews alone tipped the scale. 22 of the scores were above or equal to the required 85, and only 17 were below it. If more reviews were better than the requirement, how did they miss their average?  There is a very good answer, but be warned, it does contain math.

In situations where you are taking the mean (read: average) of a large set of numbers, scores on the extreme ends of the spectrum, called outliers, can greatly affect the total. If you are shooting for an average of 85, to counteract those two 65s you would need multiple scores that are much higher. Unfortunately, it is significantly more difficult to get a higher score than to get a lower score, especially since there is an upper cap of 100. The two 65 scores theoretically could be counteracted by one score of 125, which is impossible to accomplish. So, on Metacritic, to successfully counter the two lower scores, Fallout: New Vegas would have had to net two 100s and one 95.  To cancel two, not even negative reviews, would take two “Perfect”s and one “Amazing”. That does not seem fair at all. This creates a tendency to skew all results toward the lower end of the scale, unless you are The Last of Us and get an insane 39 perfect scores.

That’s just the math of the problem. There are hosts of other problems with this situation. Like the fact that humans generally tend to remember negative emotions far better than positive experiences, so one negative moment with a game can overcome many enjoyable ones.  Metacritic is a great reference to see a general overview of quality.  If a movie or game received a 20 and it has 50 reviews, it is fairly safe to assume it is not that great, and a 90 is fairly reliable score for a decent game.  But what about the 50s and 60s?  They definitely could still provide enjoyable experiences, but the math of averages could be screwing them out of a higher score.

While The Last of Us is a wonderful game, and I am recommending it to everyone who has a PS3, it definitely did not deserve 39 perfect scores. The game’s rave reviews led it to get a 95 on Metacritic, which is absurdly high and still doesn't say much at all. The current obsession with Metacritic scores and the extent in which they are affecting the gaming industry is preposterous.  If a company wants a meter to decide whether to pay out bonuses, there must be a better way; even if it’s the classic method of a sales goal (as long as the goal itself is reasonable. I’m looking at you Tomb Raider).  Denying a 70 man team a sizable bonus because 3 people didn't like their game as much as the others is simply unfair.

 

No author bio. End of line.
GeekTyrant Homepage