• Scope Creep - the complexity of reviewing

    In the early period of my gaming experience, if someone asked me what a game was like, the answer was simple. “It’s great. The controls are good, there is lots to do, it takes two hours to complete and it takes four minutes to load. I didn’t encounter any bugs.”

    Fast forward to today, and if someone asks me what a game is like, the answer is less simple. “It was good when I played it, but they keep changing it. I think it was rushed. There were bugs everywhere, but they might have fixed that by the time you play it. Enjoyment also depends on how much you are willing to spend on in-game purchases and power-ups. It takes 50 hours to see the main game through, so I’m only half way there, but there are an impossible number of side-quests that will keep you going till the end of time, so I can’t comment on those either.”

    So how did we get to this point? In the olden days (back when you had to get up out of your chair and press a button to change channels on your TV), you'd get a £1.99 Mastertronic game, load it, play it, tell others about it. Easy. You could guarantee that the experience that other gamers get is the same as you, as long as they had the same platform. If you are being really picky, I suppose you could say that those using a Konix Speedking might have a slightly different amount of fun. I’m all up for a bit of pedantry, but that’s really a step too far.

    Then came unoptimised 50hz PAL conversions, which were like playing in treacle when compared to their 60Hz NTSC “as designed” cousins overseas. However, if you never tried the 60Hz versions, you’d be happy enough and at least within region, one cart is going to provide the same game as the next cart.

    The next layer of complexity was the addition of online multiplayer elements, which might work great when there are only 80 reviewers plus the dev / test team playing on the servers, but it might be different for users with poor connections or servers overloaded due to lack of stress testing (or just not caring about the initial first seven days of lunacy). Near the start of the time when action games became playable over the phone, the gaps in possible connections rates were huge, in terms of ping – anyone on an ISDN connection would have a large advantage over normal dial-up users, and if using dual-ISDN, the bandwidth gap was also large.

    Additionally a major problem is the limited number of players prior to the game release making it difficult for reviewers to play the multiplayer for long enough to judge the real longevity. Also, on less popular games, the multiplayer might be great, but if no one is connecting to play online three months after launch, it makes the review dated already.

    PC games have a whole additional consideration, lots of them allowing you to choose the level of detail and / or screen resolution depending in your setup. Playing Quake II on a Pentium? 120fps smoothness for the rocket-in-the-face win.

    The next phase was the addition of enough storage on consoles to allow bugs to be “patched” PC-style. At this point, what a reviewer sees might include game-breaking bugs that are then ironed out at launch so the user experience does not match the reviews, leaving everyone wondering why the reviews were so hideous (and seemingly wrong). Conversely, a patch might add bugs to a game.

    Finally (for now) came fundamental changes to the way the game is experienced - reviewers of Forza Motorsport 5 said it's too much of a chore to get the cars you want. Then Turn10 rolled out a patch that reduced the "price" of the cars substantially, so anyone buying after a certain point will have enjoyed a fairly different experience to the reviewers and early adopters. Incredibly, the game is further divided depending on how many previous Forza titles you have caned. I did the grind for a few days and got 600k credits. Then I got a "You played Forza 2 a bit" bonus of 3 Million credits! BUT…. It was a mixed blessing. Yes, it was initially exciting to get, but it made my feeble credit earning from actually PLAYING Forza 5 seem like pittance. The end result left me quite down about spending all that time grinding. It felt like I had engaged cheat mode and I hate cheat modes. So what did I do? I bought an expensive car, realised it was rubbish and the wrong choice and played Battlefield 4 instead. Which didn’t work in multiplayer properly until after two patch releases. It’s also worth noting that Battlefield takes time to mature – the casual players drop after a couple of months, leaving a much more team focused and helpful core userbase. If you reviewed it much later than release, you’d get a far better impression of the multiplayer madness.

    So while it’s great that some of these issues CAN now be addressed, it means that the best reviews to read when making a later-than-release purchase would be the most recent ones. Which goes directly against the “Must be FIRST!!11!!” mentality that pervades the game reporting interweblogsphere. It’s good to see some mainstream sites now insisting on waiting till just after release to publish their reviews to ensure they can check the real-world performance of each game’s support infrastructure, even though it must annoy publishers intensely. A solution is to go back and update reviews – not much fun for the reviewers though, who will have moved on to the next game on their list. The solution that works for me is to check the initial reviews and then read / contribute to the Bordersdown First Play threads for the latest sensationalist comments from excited readers.

    My Google+ Profile