Data & Research

Which iPhone gaming review website marks the hardest?

The spread of averages revealed

Which iPhone gaming review website marks the hardest?
Like a French villagers making sure they use every single scrap of the fattened pig they've slaughtered, so it is with the PocketGamer.biz Quality Index.

The final bit of research from the period July to September 2009 (our Q3 report), concerns the media.

The Quality Index Best Games list and the Best Publisher list are generated from the review scores of various iPhone websites, so it's easy to invert the process and take a look at what this says about the Fourth Estate.

Scores on the doors

Simply, all I've done is taken the list of the 87 games that received three or more reviews (which enables them to qualify for inclusion in the Quality Index), and split them out for each website.

Then it's easy to total up the review scores and make an average across the number of reviews each site made during that period.

As you can see, no one reviewed all the games, although Slide To Play got closest, and Pocket Gamer wasn't far behind.



Clearly Pocket Gamer has the toughest review policy, with an average score of 7.3/10 - significantly lower than other sites.

Slide To Play, TouchGen and IGN Wireless fall into the next category with an average score of 7.7.

AirGamer, PocketGamer France and MobileGamesFAQ make up another category with an average of 8 or more.

Note: because they review fewer games, they naturally tend to have a higher average as they are more likely to review higher quality games but not lower quality ones.

The outlier is 148Apps, which reviews a lot of games, but also had an average score of 8.3: well above what you would expect considering the number of titles covered.

Note: although 148Apps marks out of 5, because it gives half marks, this is effectively the same as marking out of 10.

More methodology

Of course, there are some other caveats to point out about this data.

The most obvious is the bracketed figure for Slide To Play's average score. This is because the website has an odd marking system which rates games out of four.

Converting this into the ten scale used by most other website causes some problems as you have 2.5, 5, 7.5 and 10 as the converted scores, which - especially because Slide To Play awards many more 3s and 4s - means a straight calculation over emphasises its generosity.

Hence the bracketed figure assumes a 4/4 score converts to a 9/10 score, lowering Slide To Play's average.

Also, because the data is based on the PocketGamer.biz Quality Index, we're not covering all the games these websites reviewed during the July to September period.

However, there's no reason to suggest this would skew the results significantly. I merely mention it for the sake of completeness.

Contributing Editor

A Pocket Gamer co-founder, Jon is Contributing Editor at PG.biz which means he acts like a slightly confused uncle who's forgotten where he's left his glasses. As well as letters and cameras, he likes imaginary numbers and legumes.