We're taking on the challenge, with the PG.biz Quality Index. It's kicking off by tracking every review on five key consumer mobile game review sites: our own Pocket Gamer, as well as IGN Wireless, Mobile Game FAQs, AirGamer, and Midlet Review.
The methodology is simple: we're tracking every review in our database, and every quarter will analyse the data to see which publishers are performing well and badly, and how that's changing over time. We've converted each site's ratings into a mark out of ten.
We're starting with the first quarter of 2008, which involved 313 reviews across the five sites in January, February and March. The publisher chart below features companies with five or more reviews.
And the results show Gameloft way out in front of its rivals, with an average review score of 7.94 which is doubly impressive, considering that came from 34 reviews, by far the highest number in this analysis.
I-play (7.61), Digital Chocolate (7.60) and HandyGames (7.54) are the next publishers along, with Glu Mobile (7.44) and EA Mobile (7.30) in mid-table.
It's not so good news for the likes of In-Fusio (6.98), Disney Mobile (6.95), THQ Wireless (6.93) and Vivendi Games Mobile (6.87) who are nearer the bottom of the chart. Though we'd argue they're not disgraceful averages by any measure.
| Publisher | No. of Revs | Avg Score |
| Gameloft | 34 | 7.94 |
| I-play | 15 | 7.61 |
| Digital Chocolate | 19 | 7.60 |
| HandyGames | 10 | 7.54 |
| AMA | 5 | 7.48 |
| Glu Mobile | 20 | 7.44 |
| Hands-On Mobile | 8 | 7.40 |
| EA Mobile | 21 | 7.30 |
| Twistbox Games | 13 | 7.19 |
| Namco Bandai | 7 | 7.09 |
| RealArcade | 6 | 7.05 |
| Player X | 7 | 7.03 |
| In-Fusio | 13 | 6.98 |
| Disney Mobile | 11 | 6.95 |
| THQ Wireless | 16 | 6.93 |
| Capcom | 6 | 6.88 |
| Vivendi Games Mobile | 13 | 6.87 |
We'll have more charts pulled from the data in the next few days, including the best individual games, and the relative strictness of the different review sites.
We're keen to hear your views on this data, too. Are there other reputable review sites we should be including, for example? Do post comments or email to let us know.
Meanwhile, there are some caveats to the data: for example, when a game is published by one company in one market, but another elsewhere (for example, Townsmen 5 is counted as a Disney Mobile game, rather than HandyGames).
We'd like to run a similar table for developers, and are investigating this possibility (it relies on how accurately the various review sites report developers of games).
