EEDAR, the video game research and marketing company, has examined exactly which factors have the most impact on a reviewer when it comes to how they score a game.

Sartori Bernbeck, head of the EEDAR analytics team, has discussed his findings in an article for gamesindustry.biz, highlighting exactly which factors of games design journalists focus their critical eyes on, from “the main character’s clothes to the extensiveness of player agency and choice within a story.”

To better understand how reviewers look at things and what particular facets are kept as a focal point, the research team over at EEDAR analyzed many different game reviews across several different platforms and genre types. Once the reviews were gathered, researchers evaluated the data to identify the positive and negative areas which the reviewer focuses on for each.

It’s all pretty interesting stuff, so let’s take a look.

These areas were then categorized as one of 63 different gameplay factors, which can be divided up into eight key areas:

Gameplay: controls, level design, combat systems etc.
Market: expectations, especially in regards to controversy, sequels and market leaders
Social: multiplayer offerings and sharing with friends
Narrative: dialogue, plot, characters etc.
Graphics: aesthetics, fidelity, animations etc.
Technical: performance, bugs, physics etc.
Value: price, DLC, breadth of content etc.
Audio: voice acting, music, sound effects etc.

By finding out more in regards to how reviewers work on their reviews, EEDAR are then able to pass this info on to developers - which in turn helps them gain a better understanding of the factors which have had a strong influence on titles which are comparable to their own. For a generalisation, however, EEDAR have determined three main areas of focus for understanding how a journalist functions.

1. Reviewers tend to focus primarily on gameplay, but do critique a broad range of other game aspects.

To really understand exactly drives a reviewers opinion, you first need to understand exactly which aspects a reviewer spends the most time talking about. Gameplay discussions are often the most descriptive, due to readers mainly wanting to know exactly how a game works.

From personal experience, I can tell you that this often depends on the type of game you’re playing and how it’s marketed to its audience. For example, we all know that Telltale’s The Walking Dead is mainly story-driven, with a basic point-and-click functionality. If I, as the reviewer, would then use 400 words in a 1,500 word article to describe what is incredibly basic gameplay, and only discuss the story across 200 words, this could potentially alienate readers and cause confusion.

Although the diagram showcases that value and audio get the least amount of discussion within reviews, this again depends solely on how important these areas are and if they’re a particular drawing point into the game. A review of a Final Fantasy game would probably garner quite a large amount of discussion in regards to sound, mainly due to the expectations that fans have regarding the soundtrack, which has become world-renowned and played to audiences in concert halls the world over. An indie game which is priced above (generally) £15 would gain some discussion in regards to value, mainly due to the price expectations that readers have for indie titles. Areas of discussion rely greatly on context in reviewing, and looking at the wrong areas can cause readers to understand things as contextually incorrect.

2. Narrative aspects within games are a “pain-point” in reviews.

Looking at the positive aspects of games within reviews is an interesting one. As triple-A titles are moving further and further towards looking like actual movies, indie titles are also reaching a point where their graphics can rival some of the largest development companies in the world (just look at The Vanishing of Ethan Carter, for example). It makes sense then that graphics is the area in which reviews mainly focus on the positives, with companies often striving to make the best-looking game they can; the popularity of Let’s Play videos on YouTube could be a key factor in this regard. Social aspects, audio and gameplay also seem to be coming off in a positive light, although titles which follow the path of generic market leaders are often scrutinized for the fact.

Accordingly, in more recent years, narrative and technical areas are the ones which have garnered a more negative response, mainly due the high standards that games are held to in reviews. Whilst graphically and socially, games have managed to adapt and move on as technology has become more mature, narrative aspects just haven’t managed to keep up. Bugs and connection issues are also just some of the issues seen within the technological side of things, and we all know that this year, these issues have been nought but prominent in the industry - need I remind you of the issues Assassin’s Creed Unity and Warlords of Draenor had this year?

Does this seem like a call-out for developers to start shifting their focus towards other aspects of development over graphics and gameplay? Potentially. It sound obvious, but reviews have to have some mention of gameplay within them, because a game has to have some form of interaction between the player and the game. So developers tend to make sure that gameplay is polished due to the forgone conclusion that reviews will mention this aspect.

Other elements then fall out of priority, because do players really want a complex story in a platformer, or are they going to be more bothered about whether the level design is interesting? Is music a defining factor in an FPS game, or do players want beautiful graphics that look just about real? Some of the best games to have come in recent years have tried to be innovative in this regard, and succeeded. Others simply fall flat. Publishers rarely want to take chances, and it unfortunately means that the general consensus is that a game needs to play well and look pretty to be successful.

With indie developers coming into the spotlight a bit more, however, it seems that these issues are slowly being resolved. Take Minecraft, for example, arguably the most successful game of the past five years. It’s not particularly pretty, and to this day still has a fair few bugs, but people love it. Maybe if developers were to take chances a bit more often in these other areas, we could further rectify video games to the general media as an artistic medium, and not just viewed as money-making cash grabs that are produced annually as a way to keep teenage boys happy.

3. Single player ultimately drives the score given on a review, even for games with a strong focus on multiplayer

Although everything mentioned prior to this is considered to be the areas which are considered to be the primary focus of review writing. But what areas of development actually impact the review score itself the most?

In general, the biggest defining factors are the comparisons between the products already in the market, previous iterations within a franchise and how well a product does regarding its own expectations. Gameplay and narrative also have a strong impact on score, but interestingly enough - and despite being the third most discussed topic in reviews - multiplayer and social aspects simply don’t have a hold on a score in the same way that other areas do.

Even with games that have a primary focus on multiplayer aspects, reviewers tend to take an explanatory approach in regards to covering them, explaining how the multiplayer functions and why it is there. Simply put, this is for one main reason: it’s difficult to establish how well multiplayer aspects function when there’s no solid community there to see how it plays. It’s one of the reasons that Halo: The Master Chief Collection did so well in reviews. Prior to its release, there was no telling as to how the multiplayer would function, and even today - three months after its release - it is still having trouble. Yet, the single-player is fantastic. I’d personally never played Halo before the release of MCC (I know, please don’t judge me) but the single player campaigns have helped me to realize why it’s regarded as one of the greats.

Both as an aspect of this point, and collectively, it makes for some interesting discussions to be had. Eurogamer have recently dropped review scores, with the belief that “scores are struggling to encompass the issues that are most important to you.” Oli Welsh continues in the article to ask “How should we score an excellent game with severe networking issues? A flawlessly polished game with a hackneyed design? A brilliantly tuned multiplayer experience with dreadful storytelling? If you expect the score to encompass every aspect of a game, the task becomes an exercise in futility. Add an inflated understanding of the scoring scale in many quarters… and you have a recipe for mixed messages.”

It’s an interesting approach, and opinions are very much divided. Some people like the idea of scrapping scores, omitting the need for a numerical value to define a game, others view it as a simple alternative that seems a lot more vague than a review score. Either way, it’s a move which will most definitely be interesting to see how it pans out.

Currently here at Continue Play, we operate on a rating system which defines a game with a score out of ten. You can read more about that here, if you’re really interested about what each rating means. Even so, we’ve been having a lot of discussions about the formatting of our reviews, and how we should move forward over the previous weeks and, indeed, months. In fact, we started working on what we view as the eventual future direction of Continue Play around Summer of last year. We can’t discuss it yet - it’s still relatively early days - but we have a clear view of where we’ll be heading in the future. In the meantime, other sites such as Kotaku, Joystiq and Eurogamer have all revealed radical new policies over the last few months about how they approach reviews and content, which, of course, has led to certain people trying to claim a “victory” for things which had absolutely nothing to do with them, and were often already in the works for months.

We haven’t come to a clear decision as of yet - and we’ll let you know when we do - but we still stand by the same policy which we always have done: Our reviews are subjective. They reflect the views and opinions of the author, and we take pride in that we will NEVER be swayed in our reporting or content by outside pressure.