What’s Up With S.T.A.L.K.E.R. 2 Reviews? (Patreon)

I originally set out to write something specifically focused on the Stalker 2 review situation, but as I found myself needing to explain more and more about how game reviews work and some of the annoying things that come with them, it became a bit more general and, well, longer.

S.T.A.L.K.E.R. 2 reviews are in, and it’s easily one of the weirder review cycles I’ve seen for a big game in years. That’s not me saying that the reviews are invalid or that the game’s publisher has done anything wrong, really. It’s more that the entire process for reviewing video games in a timely fashion isn’t especially compatible with the way today’s games are made. As a result, reviews have been all over the place or, in the case of some publications, they’ve decided to hold a review completely and wait for the dust the settle. What the hell happened here?

Let’s back up a little bit. Reviewing video games in a timely fashion is rarely done in an ideal setting, especially these days. In a “normal” situation, the main pressure is time—a reviewer is assigned a review and, if they’re lucky, they’ll have about a week to get through a game, write something up, get it edited, posted, and all that. Maybe they’ll have to adapt their text into a script for a video review while they’re at it. For most games, this is usually plenty of time, assuming said reviewer doesn’t have much else going on. One might argue that the very idea of saying “hey, here’s a game that’ll take you something like 40-50 hours to complete” seven days before a review publishes is already a bad situation, but let’s save the labor discussion for another time.

In 1996, when I first started reviewing video games for a web site, the process was blissfully simple. Console games went gold, meaning they were done, complete, finished, immutable. But because of the particulars of manufacturing physical discs and getting them shipped to stores, console games would go gold like a month before their actual street date, if not more. This means the song and dance of getting a review copy was less a question of the game actually being finished and more one of routine. Companies sent out games for review on CD-Rs, we reviewed them, we moved on. Sometimes a game wouldn’t show up and you’d need to have gentle conversations with publishers and public relations people in an attempt to figure out why. Was the game’s release date slipping at the last minute? Was the publisher certain that the game wouldn’t review well? Had they negotiated an exclusive with another site that made them, for whatever reason, not ship the game out to everyone else until much closer to launch? It was a short list, really, and in a world where a lot of these games were out in Japan months ahead of their domestic debut, parts of the process felt more like a formality than anything else.

The PC side of things, even back then, was a little more wild, a little more woolly. Most games didn’t have auto-patchers, so you’d have to go and seek out a patch, download it, and apply it to a game. But since PC games could be patched, that led to more publishers pushing out a 1.01 patch at or around the retail launch of the game. Typically these were small changes to fix incompatibilities with specific sound cards or whatever, as opposed to fixing big, sweeping crash bugs. So if you got the game on launch day and it worked, you’d probably never even think to look for a patch in the first place. Still, the PC editors at GameSpot back then did what they could to ensure they were reviewing the most recent version available at the time of review, and they had an entry that appeared on all their reviews that listed the specific version reviewed, just to clear up any confusion.

All things considered, these were “the easy times.” Nowadays, obviously, the situation has changed and there are way more moving parts. For starters, console games can now take patches just as often as their PC counterparts. The idea of a game being “done” has more or less gone out the window, giving rise to the “day one patch.” Originally, this meant that a game was still largely complete and shippable in its “gone gold” form, but the team would keep working on the little things and hopefully have the game fully complete by the time it was in your hands. Over time, it feels like some games have leaned on this process a little too hard, resulting in a shipping version that feels a little bit less like a finished product. This, however, gets offset by the increasingly digital nature of video game purchasing, meaning that the game that’s on the disc matters less and less because more and more players are just buying it digitally. They’ll get whatever the latest version is when they buy it and they’ll stay current because now games just patch themselves.

It must’ve been a bummer for the GameSpys and the FilePlanets of the world when players no longer had to go hunt for downloads on sites covered in ads and offers for “premium download speeds.” But here in the now, it’s actually harder to not patch a game than it is to just let your Xbox or PlayStation or Steam or Epic Game Store automatically keep your games up-to-date. As someone who used to have to ask other players if their Warcraft II client was patched to the right version so we could all play multiplayer together, this is a positive development. And yet…

A lot of this comes back to the knock-on effects of these moves. As the pipelines for keeping players fully patched have been developed, the actual quality of the average shipping game feels like it’s slipped. Think back to extreme cases here, like the version of Mortal Kombat 1 that shipped for Nintendo Switch. Without its day one patch, the game looked like a parody of itself and was missing major features, as well. To be fair… the patch didn’t exactly fix everything wrong with that one. But it’s still a solid example. Here’s the thing, though: none of that should actually matter to the average consumer. Because the average consumer will put in a game, automatically apply a patch, and get whatever the “good” version currently is. At most we’re talking about a complaint around download speeds or bandwidth caps as opposed to an actual problem with the game.

But we’re not talking about the average consumer here. We’re talking about the review process. Now go back and think about that reviewer, who used to just get a CD-R in the mail, plop it into a debug unit, and get to work. Now that disc is a Steam code and it may very well come with a long list of notes about the game they’re about to review. Some of these are totally benign, like a note from the producer, thanking the reviewer for taking the time to play the game and talking about what a long road it’s been to get that moment or whatever. Sometimes you’ll get lists of “known issues.” This is where things get tricky.

A known issues list—meaning a list of bugs that the developer is aware of and is looking to address before launch—muddies the water almost instantly. While the bug list is provided in a way that can help reviewers (sometimes they’ll include workarounds for soft-locks and stuff), it’s also there to discourage reviewers from focusing on a game’s technical issues. After all, why write about it if it’s just going to be fixed before launch, right? In a perfect world, that would always be the case. The bugs would get fixed, the game would ship without issue, reviewers wouldn’t waste time focusing on things that regular players will never see.

But what happens when the bugs don’t get fixed? I’ve been on both sides of this over the years. I’ve had cases where I mentioned bugs that were gone by launch—some of which were never mentioned on any kind of pre-release information email—and I’ve had cases where I’d seen bugs, but maybe not enough or not bad enough to call out specifically, that ended up being a bigger deal for more players post-launch. It’s a shitty situation for everyone because you, as a reviewer, have no way of knowing if a developer will actually fix those bugs before launch or not. So that known issues list really just ends up feeling like a flashing light that says “wait for retail.”

Of course, having some kind of coverage at the embargo or release date is often way more desirable than sitting around for a week to check against the shipping version of a game. It’d be nice if this weren’t the case, but for as long as I’ve been doing this, the interest level (meaning "traffic potential") around a game review spikes just before its release date, and then immediately falls off. You very quickly enter a territory of “well, if we can’t have it at the embargo, then we shouldn’t bother with it at all” at some of these larger, more numbers-focused outlets. And, in not reviewing a game, particularly one that has the sort of issues that readers absolutely need to know about before making a purchasing decision, the publisher of the game kind of wins by default. This was especially true back in the GameSpot days, where we’d have a year or so of neutral-to-positive preview coverage only to learn, whoops! The final product kinda sucks. Hey, it happens!

So what happened with S.T.A.L.K.E.R. 2? A couple of things happened, one of which doesn’t really happen very often at all anymore and one that I can’t remember ever seeing before in my decades of doing this.

On the front end, the information accompanying the review copies mentioned technical issues. But instead of listing them out, it simply said something to the effect of “anything you encounter during your review is being fixed and we’ll be patching the game as you play it.” So instead of a known issues list it became an… unknown issues list? Along with the sort of empty reassurance (we’re workin’ on it! All of it!) that no reviewer should be taking at face value. Here’s the actual text:

Technical issues:

Please be aware that the review copy will be updated as you play, and we have a day-0 patch in preparation, which will be released simultaneously with the game. All critical bugs and a fair part of minor ones you may encounter are currently being addressed. With the day-0 patch, players will experience a more polished version of the game.

This isn’t unheard of, I’ve seen games come with this kind of thing before, but I couldn’t tell you the last time I saw it. It’s the sort of practice that went out of style in favor of a more spelled-out list of issues, so you’d know what to look for. While I don’t doubt that this was the intent of the developers—of course they’re going to keep working on bugs—it comes across rather poorly to just say that anything you encounter is being addressed. Like, oh, are reviewers just supposed to ignore everything they see or hear? This ended up being compounded by the game itself, which was an absolute mess when it first went out to reviewers.

I’ve gone into this, but to briefly recap, the PC version of the game crashed, it had weird performance issues, and a situation where the player’s guns didn’t make noise when fired. Radio messages from characters would drop the radio effect halfway through a sentence, making them suddenly sound like they’re right there next to you. And that’s just stuff that I saw. Sounds like other reviewers had worse times with the game while others didn’t encounter much at all. That’s the nature of bugs! They’re weird! They don’t happen to everyone!

Over the course of the week, the game got patched a handful of times. Sometimes this would correct stuff (eventually the guns started making noise more reliably) while others seemed to be introducing new performance issues. Now envision someone trying to put 35-40 hours (their stated time for a “core playthrough”) into a moving target. Frustrating, right? The last pre-release patch for the game seemed to land the day before release.

On the morning of launch, a follow-up email was sent, highlighting all the bugs that had been fixed over the course of the week. The public relations rep sending the email claimed that 1125 issues had been fixed over “the last few days,” including 111 crash bugs, 70 “priority 0” bugs, and so on. Things like the weapon audio were called out by name along with some other soft-lock scenarios and visual adjustments. This would go on the be the launch build of the game. The radio audio thing, as far as I can tell, still hasn’t been fixed and the developers have since published one of those social media letter screenshot apology things where they thank everyone for playing the game and remind you all that they’re still working on it.

And life goes on. The game certainly still has issues, but so far it doesn’t seem like the issues are as major as they were a week ago. Most of you will never see the worst of it. And that’s a good thing! But it also means that all of these reviews that have been hitting at or around launch are… weird. They were written in less-than-ideal situations. Really, one of the weirder review cycles I’ve seen in ages. But also I don’t know what else could be done here.

But think of the poor reviewers, right? Nah. Look, I am a person who still puts value in reviews and purchasing recommendations and all of that stuff. But there’s no “right” way to handle this on the publisher and developer end. If they don’t send out review copies at all, then the public gets suspicious about the game’s quality. If they send out review copies the way they did, they end up with a bunch of reviews dinging them for technical issues that might not exist anymore because, realistically, you can’t push a final patch 24 hours before a game launches and expect reviewers to play your “35-40 hour game” all over again in that period of time.

On the reviewer side, the answer is even murkier. The actual right thing to do here is to wait and see what the game is like when it ships. This answer has been unpopular with the bean counters and business types for as long as I’ve been doing this, and the number of times I’ve had to talk down people in those roles when we were deliberately going to be late with a review is uncountable. Some outlets have started producing the “review-in-progress” piece, which is pretty much an unscored version of the review that you update and slap a score on once you’ve had time to verify everything.

Editorially, I think this works, but it’s certainly less exciting (and probably does less traffic) than a proper review. But it also means that you’re taking a reviewer and now locking them up for, what, a second week? So they can keep messing with the game and see if the bugs were actually fixed? Is that a good use of anyone’s time?

Ultimately, I think the answer here is that there aren’t any good answers. I don’t feel like anyone out there has gone through this process in an intentionally deceptive way or acted with any malice, but with review scores all over the place and a situation where some reviews spend a ton of time covering a litany of bugs while others don’t, I felt you might benefit from a little perspective on how we got here and, to an extent, a long view of precisely this sort of thing and how it happens.

My recommendation for S.T.A.L.K.E.R. 2 is that you might want to wait a week or so to give them time to iron out some more issues before diving in. But considering some people view “jank” as a core component of the franchise, maybe you should just give it a shot now, especially if you’re a Game Pass subscriber since you don’t need to spend any more money on this kind of thing.