One of the threads in the Game Journo Pros private e-mail discussion group between media journalists, press representatives and website owners for tech and gaming websites was talk surrounding the review scores on Metacritic costing Bungie $2.5 million in the bonuses they were promised if they hit a certain Metacritic average for the 2014 release of the game Destiny. The discussion centered around Metacritic review culture and the power reviewers (and their scores) have on costing developers large sums of money, the kind that kind make or break a studio’s future.
It starts off linking to a Kotaku article that was making the rounds about Destiny‘s review aggregate failing to meet the 90 that Activision arbitrated in the contract for Bungie to receive the $2.5 million bonus.
Most of the comments in the thread acknowledged that the game was purely mediocre and deserved the score it received, while others lamented that review scores have such sway and power over major blockbusters from publishers.
That last line about Hamlet… that was from New York Times contributor Harold Goldberg. When he mentioned that Bungie and Activision were looking for him to do a special feature at the New York Times, I decided to reach out to Activision directly to see what sort of feature they were expecting from Goldberg.
According to Evan Bader from PMK BNC, the firm that represents Activision and Bungie for Destiny, he denied that there was any special request to Goldberg for the coverage, stating…
“As a PR agency we work with a number of journalists” […] “We had the fall preview and the review that Chris Suellentrop wrote for the New York Times, they are both on the [NYTimes.com].”
“To my knowledge, I provided Harold [Goldberg] with a review copy.” […] “I’m not exactly sure what the other feature he is alluding to.” […] “His [review copy] arrived on [September 10th].” […] “As always, we mail out like 100 copies of the game to reviewers as well. So it just comes down to if they write a review, great, if not… no way are we expecting them to guarantee us that they are going to give us a review.”
I reached out to Harold Goldberg a couple of times to ask if he would clarify what he meant about Bungie and Activision expecting a special feature, but over the past couple of months Goldberg opted not to respond.
James Fudge, managing editor from Game Politics and Timothy J. Seppala, a contributing editor from Engadget, both politely declined to comment. Interestingly, though, Seppala had encountered the issue of review scores and developer bonuses being restricted due to review scores on another occasion, writing in the GJP thread…
“[…] this isn’t the first time something similar’s come up. There was the time that Glen Schofield called OXM out by name (and the reviewer Meghan Watt, iirc, who now writes for Ubisoft) for giving the first Dead Space a low score that cost them bonuses.”
I did reach out to Kyle Orland, the senior gaming editor from Ars Technica and creator of the Game Journo Pros list about this relationship between game journalists and publishers where scores can be altered or measured by the relationship between the two parties. According to Orland…
“In general, I believe game reviews shouldn’t be scored — it distracts from the text and makes what can be a nuanced and subjective opinion looks like something that’s more objective and quantifiable down to a single number. Trying to compare scores between different games, genres, reviewers, platforms, and review outlets is a fool’s errand, and inspires fanboy debate more than actual understanding of what makes a game interesting or good.
“Scored or not, though, I believe a reviewer’s only duty is to accurately present his or her opinion to the reader. I know many publishers and developers use Metacritic averages to determine bonuses, and I’m pretty firmly against that practice (more details on why here: ) But the decision to do that is on those developers/publishers, and I don’t think reviewers can be reasonably held responsible for how others use their scores.”
Orland also shared a link on Ars Technica about why linking developer bonuses to review scores is bad for business.
A previous thread in the Game Journo Pros list linking to an article on Cracked about what would cause the gaming industry to crash also dealt with the dangers of Metacritic and review scores. One of the topics from the Cracked list was on publishers gaming the review system with what is pretty much labeled as bribery through ads and close relationships, with the Cracked author writing…
”Basically, the publishing companies are paying the review site’s bills by buying ads and handing out free publicity, so from the struggling writer’s perspective, it’s bad business to give bad reviews.”
Some of the members of the Game Journo Pros took offense to this aspect of the article, with some of them stating the following.
Garrett Martin, the games editor for Paste Magazine, took great offense to the postulation from Cracked, responding with the following.
Michael Futter, the news editor at Game Informer was more concerned with the Cracked author’s conflict of interest.
Nevertheless, the idea that reviews or scores aren’t influenced by publishers or for page hits is seen by some journalists as absurd.
An editorial over on VGRHQ came under a lot of fire for some critics talking about the corruption of the review business and how some sites fix scores for hits. The critics made the comments under anonymity, of course. One of the anonymous critics wrote…
“The only way any site survives and the only way anybody gets paid is via ad revenue. This is why the overwhelming majority of game journalists only receive what equates to part-time pay. I was always okay with that, until I was told how I’d be generating revenue. I had one editor tell me flat-out that the site needed a boost one month, and I needed to give a big-name release a low score.
“He even said he’d post it before the embargo because as everyone knows, early reviews get a huge amount of attention.”
I did manage to reach out and get in touch with two of the critics; they were willing to provide quotes, some background info on their careers and their names, but only under the condition of their first names being made public. Community manager and long-time journalist for more than 15 years, Brian, stated…
”I was part of a site that stopped receiving review copies from a certain publisher. My editor told me it was because we gave one of their big games a really low score, but they claimed it was because we hadn’t issued a fair review. I didn’t write the review in question but I know who did; he claimed to have finished it before reviewing.”
“I think the site eventually got back in the publisher’s good graces but I don’t remember how or when. I do know that for the most part, publishers aren’t in the business of totally blacklisting a major (or even medium) website because of a low review score. Sometimes I think it’s more about the PR people surrounding the project as opposed to the publisher.”
Entertainment writer and critic since 1979, Jared, chimed in to state…
“Like Brian, I’ve never been part of a site that has been blacklisted by a publisher. Blacklisted sort of implies that you did something wrong and the publisher just won’t deal with you anymore. I’ve heard of it happening but nowhere I’ve worked. However, I’ve gotten a nasty email from a PR rep after a bad review of a game, which seemed a little weird, as that person obviously had no clue about video games. I don’t think there’s much evidence to support the idea that publishers respond differently to good and bad reviews of their products, unless they think you screwed up or were unprofessional somehow. And that’s a judgment call on their part.”
“Nasty e-mails” from PR reps isn’t uncommon, it’s something game journalist and critic Liana Kerzner also faced, and more, stating…
“My problem is more denial of access. I’ll contact a company about a story – it doesn’t matter how soft – and they just draw out the process until I’ve missed my deadline.
“The only company I’ve had specific problems with regarding retaliation for negative coverage uses their fanbase as a mob. They let raids form on their message board then their PR people stall while the fanbase eats a reviewer or columnist alive based on misinformation.
“But because of that, I don’t want to mention their name. I’m already having enough trouble getting work these days. It’s a bad time for anyone who didn’t condemn GG.”
However, reviewing games with scores that affect the bonuses of developers and corruption on the journalist’s side of the fence has been going on long before #GamerGate got underway.
According to freelance writer, event specialist and journalist Hasan Ali Almaci, he stated…
“I think I mentioned before that over the course of 15 years I only did 3 reviews. That being said they do not ask for certain scores the reviewers can decide on those themselves. They do however sometimes have stupid shit in the NDA´s that state a review can be published sooner(increasing pre release hype and preliminary Metacritic score) if the game gets a certain score(like EA did for Crysis 2 then dropped for Crysis 3 ).
“What happens more often however is publishers requesting certain reviewers to review their games because they know from precedent that said reviewer will like the game. Before being fired and starting Giant Bomb for instance Activision requested all reviews of Tony Hawk games to be handled by Jeff Gerstmann because they KNEW he was a huge fanboy for Hawk and those games.
“Back when it still existed Gunk magazine in Belgium did sell a few scores(Frank Molnar was editor and chief and pretty much is a laughing stock in this industry), cover and a high score for Catwoman. The public was outraged so they quit selling scores but continued to sell covers(he admitted as much to me)”
I did reach out to Frank Molnar, the former Gunk TV editor-in-chief and current host of the Dutch gaming site Game Nuts. Unfortunately, the conversation regarding the relationship between journalists and publishers was off the record. Regarding the allegations of bribery and selling covers and review scores for money, Molnar declined to comment.
Nevertheless, even Ars Technica’s Kyle Orland feels as if selling reviews and fixing scores is bad for business, saying…
“Imagine a world in which reviewers raised their opinions or scores so developers wouldn’t lose a bonus or get fired. The result would be bad games (as judged by the reviewers) getting higher scores, and presumably more sales, while developers who work on those bad games are more secure in their jobs. Who is better served in that world, exactly?
“Opinions on individual games are often going to be contentious, but serving up an honest personal opinion to one’s readers has to always be the only concern of a critic.”
We have seen how even good developers with well received games can still suffer at the hands of these scores. Most notably was Obsidian Entertainment with Fallout: New Vegas, where they were infamously one point short on the Metacritic score and failed to receive bonuses to keep many members of the studio employed, as reported by Escapist Magazine.
I did manage to reach out to Chris Avellone, legendary writer and game designer who works with Obsidian Entertainment, and I asked if the review scores have affected Obsidian to the point where they hire or fire based on a system that put many of them out of work, and if they put a lot of effort into the Metacritic culture that affects gaming. According to Avellone…
“We don’t, although we have paid attention to player review scores for mods some designers developed (along with other metrics for mods – downloads, number of patches, etc.). In one instance, the player review score, total downloads, patches, and then re-releases almost guaranteed the hire – the last step was just to see if the modder was a dick who didn’t play well with others. He wasn’t.
“Usually, my issue with Metacritic is that the review scores across sites can be wildly inconsistent (50% being average for X publication, for example, which can certainly torpedo a score).
And even if sites do take on a hostile nature against the studio, similar to what we’ve seen with some developers aligning with #GamerGate and coming under ridicule from some gaming outlets, Avellone stated…
“Cutting off contact, [in my opinion], can prove to be a bad thing, regardless. We’ve found in the past that seemingly hostile sites, one in particular, was very negative on Torment before they played it, and as far as I’m concerned, their negativity actually helped us on release because the game seemed to defy expectations even more. To be fair, I actually thought that (considering he hadn’t played the game), the reviewer’s comment that it was just a Baldur’s Gate knock-off was probably an impression many people had, considering the engine the two games shared.“
According to Liana Kerzner, this kind of goodwill-nature from the development and publishing side does resonate enough in the gaming industry that everyone can afford to shrug their shoulders and move on in their life. From time to time it’s not always about blacklisting and journalists being pressured for good scores, even when “bad” scores roll in. Kerzner stated…
“I will say that Ubisoft as a company has never punished a site I worked for for a low score. They gave us incredible access on Splinter Cell Blacklist, but the site I was at at that time didn’t allow us to review the game if we’d done advance coverage of the game. So the guy who reviewed the game decided to be an asshole and gave the game a six. Me and a colleague both thought that was unfairly low. But Ubi never complained.
“I figure it’s only fair to say when a company has been really cool too, yeah?”
Journalists and publishers can get along, at times. However, at the same time it’s not always so innocuous.
Hasan Ali Almaci mentioned…
“Selling scores and reviews does happen but its the smaller almost unknown sites, mobile review sellers, smaller Youtube channels and smaller magazines that make themselves guilty of that and often only because otherwise they cannot survive or even if they can they will not receive the pre and review versions they want unless they play ball. From my experience the big sites and mags do not but are still stuck in this cat and mouse game of keeping the public’s interest in mind while trying not to piss off the big publishers too much.”
The smaller sites Almaci is referring to are sites that extort smaller developers for positive review scores; sites like Appcraver and iPhoneAppReview.com, which were highlighted in a case by Indie Game Mag. These sites were found asking developers for money in exchange to give them positive review scores, showing that sometimes the reverse can be true in the money and review game between developers and reviewers.
But Almaci’s mention about the big sites trying to play the cat and mouse game not to piss off the publishers usually comes in the form of lavish parties, strip clubs and something described by Gamer.no journalist Jon Cato Lorentzen as a “booze cruise”, all in an attempt to get gaming magazines and outlets to output positive scores.
As recent as the release of Crysis 3, there was a controversy involving Electronic Arts communication manager Devin Bennett taking journalists out on a wild night on the town on EA’s dime at the review launch event of Crysis 3 in Frankfurt, Germany. According to Gameland Magazine’s editor-in-chief Govorun Konstantin…
“you should know that EA was shocked by the bill and stopped covering drink expenses other than soft drinks and beer for all press.”
I did check in with Bennett to confirm if he was there during the Crysis 3 event in Germany and he simply responded…
“The review event? Yes, I was.”
When asked about the boozy night out, Bennett declined to comment any further.
However, another games journalist was there to further detail the event. Former Score Magazine writer and current editor for Europe’s Wargaming.net portal, Adam Plechatý, stated…
“Yes, I was indeed there. I remember, because it was my first “review event” and also the last one (thank god).
“I thought it was a big pain, honestly. As a reviewer, I only cared about playing the game in peace and then writing my honest opinion on it. All those dinners and developer sit downs just got in the way and I remember playing the game until 4 in the morning just to finish it. Luckily the game was kinda short…
“I can´t really comment on the events you describe as I never took part in them. I remember having a dinner with the other journo guys and some of them had little problems spending a lot of money on EA´s bill. But I guess that is up to everybody´s conscience… Certainly nobody pressured them to overspend that much.”
“Up to everybody’s conscience” was something that Kill Screen co-founder and freelance writer Chris Dahlen mentioned to Game Informer news editor Michael Futter when Futter questioned the ethics of someone working on a game and then writing a puff piece about it without disclosure.
Nevertheless, the lavish parties and expense trips to tempt reviewers into shelling out a good score is nothing new. Back during the SimCity fiasco journalists were given special treatment for the always-on city-building game that didn’t reflect a real-world user setting, resulting in a lot of early, puffed up review scores. The issue was debated behind-the-scenes by tech and gaming journalists, but it did little to alert early customers about the game’s problems.
To Polygon’s credit, they did fluctuate the score to reflect the state of SimCity, but as pointed out later on in the Game Journo Pro thread, the first score Polygon put up is what Metacritic has recorded, which is why Polygon’s 95 out of 100 review score is still listed in the “Positive” section of SimCity’s reviews over on Metacritic.
I did reach out to Metacritic a couple of times for a comment on the issue of reviewers fixing scores higher or lower based on outside influence and if they have any safeguards in place against gaming system, especially since developer bonuses and a lot of jobs are tied to this system, however they opted not to respond.
David Jenkins from Metro tried to argue that game journalists and the review scores don’t matter much in the grand scheme of things while interviewing SCE Europe president Jim Ryan, stating…
“The one thing that does make me laugh about GamerGate is their assumption that any publisher would bother to bribe a journalist. Nobody listens to reviews, they never have and never will. Which makes me curious as to how does Sony view the specialist media nowadays, especially now with the rise of YouTube celebrities?”
Well, if that were the case the SimCity review event fiasco never would have happened, nor the Crysis 3 review event, nor would Harold Goldberg have mentioned about Bungie and Activision requesting a feature for the New York Times.
He does make a good point about the rise of YouTubers, and although TotalBiscuit briskly swept away any notion that publishers have attempted to get him to start up a website for the express purpose of scoring games so they can be added to Metacritic, they have tried swaying YouTubers in other ways. Brand deals and shady contracts, as highlighted with the Shadow of Mordor case, which actually required TotalBiscuit to step up and expose the deal, otherwise consumers never would have known that a lot of the Let’s Play material for the launch of the game was paid advertisement from a brand specialist.
Nevertheless, in the case of Destiny, Dennis Scimeca, from the Daily Dot, mentioned in the Game Journo Pro thread that it was the publisher and Bungie’s fault for leading people on, and not the journalists for getting on the hype train, writing…
“I can’t remember another game over the past four years that was so plainly deceptive as to the kind of game it was.
“Saying the game is quality when it sucks? Of course. Going to lengths to hide the fact that Destiny was a dungeon crawler? Not cool.
“Everyone knows Diablo III is a dungeon-crawler when they pick up the game. Activision and Bungie let no one know the game was a dungeon crawler until after release.
“Bully on them, it was a smart corporate move, because how many people would have avoided this game entirely had its true nature become known before it hit the market?
“This still bothers the hell out of me, though. Like I wrote in my review, if anyone other than Bungie had released a game based on such blatantly deceptive marketing, people would be losing their minds.”
At the end of the day the responsibility and onus of honesty falls to the journalists; it’s up to the narrative thread weavers to make the moral decision to stand up and expose corruption where it happens as opposed to slinking back and confiding in cohorts and peers, doing nothing more than privately echoing thoughts of ill about a culture fostered in corruption.
This is something that even Wargaming’s Adam Plechatý commented about, stating…
“I think what people, readers, need to take away from this is the fact that journalists should have the moral strength to not let their judgement be clouded by any such circumstances. Yeah, they put you in a nice hotel (you have to stay somewhere), they wine you and dine you (you have to eat something), but that doesn’t mean you have to give the game a better score or overlook some of its problems. If you do that, you are a bad journalist – publisher is just doing its job, I wouldn’t call it a bribery.”
It doesn’t matter what it’s classified as, when clear conflicts of interest arise and consumers suffer at the hands of behind-the-scenes politics and backdoor dealings to puff up scores, partake in paid deals, lower scores, take money in exchange for selling covers or fix scores, it should be within the interest of those adhering to ethical standards to come forward about the issue.
The Metacritic game may be a complex issue between both publishers and journalists, but consumers should always come first.
If a YouTuber like TotalBiscuit can disclose conflicts of interest, then so can every other writer out there that takes on the label of being called a journalist.