This week's question was prompted by Graeme Devine of GRL and his GDC Online rantthat measurement and business micromanagement were distorting game design.
So we asked the Mavens; Is our focus on metrics and monetisation killing mobile games?
Initially, the reaction was negative about both metrics and monetisation.
Neil Holroyd of Everything, Everywhere was first to respond; "A lot of people try to get rich quick and aren't really thinking about the long term effects and opportunities to grow with your users."
Kyu C Lee of Gamevil took a more philosophical approach. "True, there always have to be a balance and it's just another approach. Taylorism was born in the manufacturing industries. And in physics, there are theoretical physics and experimental physics, which provide two different approaches."
Mix it up
"Metrics serve their purpose, but should not be allowed to kill innovation or to keep you from taking some form of risk with a game design," thought Andreas Vasen of Machineworks.
"Plus, there is portfolio balance. After a successful line of metrics-driven games, build something crazy!"
Wen Chen of Coconut Island was more discerning. "The focus on monetisation is killing mobile games. Metrics? It depends. Using metrics of users' gaming behaviour to improve a game is right but the goal should also be to make the game more fun to play instead of grabbing more money from the players' pocket."
Mills of ustwo, whose Whale Trail games seems to be about to scupper his self-proclaimed title 'King of Fail', was adamant he wouldn't touch metrics or monetisation.
"The focus has to be on creativity, playability and usability ... money should never, in my absolutely nothing opinion be the driver for decisions."
He was similarly scathing about metrics; "The thought of trying to make something engineered to explore the psychological touch points that coerce humans to part with their money ain't our bag.
"The day I use metrics to design our interfaces or games is the day I give the fuck up. Life's too short to make things for others.. Makes something special that you want, not what the majority of humans think they want. Feed a dog dog food and it will obviously eat it. Feed it steak tartare and it will give your dog a bone."
Black or white
But beyond the hyperbole, he was clear what made a successful game. "You're either creative and innovative or you're not. And that means the whole package. The bottom line is most games bomb on the App Store because they are utterly terrible, metrics or no metrics."
Tower Studios's Jon Hare was equally scathing. "Metrics are another tool arming the non-artistic with the ability to dominate the sensibilities of the artistic. Metrics, in-game advertising, and freemium games, this unholy trinity are probably nothing more than a current fashion and a reflection of the over-commercialised market that currently exists."
"The idea of relying on metrics to do your interface design for you is also amateur. In fact, creatively it shows a colossal lack of talent for anything other than me2gaming. Metrics are teaching you how the dumbest (i.e. the majority) of people play games, as well as how the brain responds to certain patterns, layouts, emphasis and reassertions."
Typically outspoken, Kevin Dent of Tiswaz thought games without metrics were great; if you designed them with metrics
"...You absolutely should become an accountant... mediocrity is and should be cast aside with disdain. For the most part, studios use the freemium business model as a crutch to produce average or subpar content," he argued.
Tool for the job?
Then, for a moment, it seemed he was about to leaven his message.
"Let me qualify that," he said. "If there is any other measure in a game than your performance regarding a win or lose scenario; you are selling out not only the player, but your studio. In my mind though, metrics can be useful after the fact, but only in terms of balance."
He noted thirdparty metrics tools were dangerous. "The ramifications for SDK bloat are profound and they will destroy your game or app. If you decide to go the metrics route, fine; but don't screw the objective, which is to ultimately entertain."
Dave Castelnuovo of Bolt Creative thought money was secondary, but it was an easy, clever trap.
"By following the market, you can perform at some acceptable minimum but the only way to reach a breakout hit is by developing something for yourself. I suggest watching the Steve Jobs' Stanford commencement speech again, 'Follow your heart and intuition, somehow it truly knows what you want to become, everything else is secondary'."
Cookie cutter approach
Despite this, he was aware that metrics were of use, and admitted to using them himself.
"It really depends on your personal style. If you enjoy number crunching, then have at it ... Once in a while I check to see which operating systems and devices we can cut support on."
Volker Hirsch of Blue Beck and Scoreloop agreed - to an extent.
"If you try and wring something out of a pseudo-scientifically-composed metric *only*, it is unlikely to fly. You do need heart, blood, creative excellence, passion, and pure loving gaming goodness.
"Proof? Plants vs Zombies. The game was legendarily long in creation and most people would have dismissed it on pure metrics terms - I mean, come on, the undead and bloody veg!?"
Believe in your talent
Castelnuovo also believed all developers want to create something culturally important, and not have to work for hire ever again.
On that basis, he said, "I believe that by looking at the numbers and monetisation of the app, you can decrease your chances of losing. I totally agree with that. But I also think it lowers your chance of hitting the jackpot."
He agreed with Plants vs Zombies as a counter-metrics success while pointing to Gameloft as being metrics-driven; a company that builds profitability by borrowing other's originality.
Finally, he pointed out that there wasn't a single solution to making money whilst making great games.
"That guy from Wired [Kevin Kelly] who wrote about how 'everything will be free' told a little story about successful survival strategies: some organisms have a small number of children and have to protect them carefully for their species to survive, while things like dandelions let off thousands of spores with the idea that only 1 out of 1,000 actually need to bear fruit.
"His thought was that freemium was like the dandelion and that if you get millions of free apps in the hands of users and only 1 of 1,000 actually pay money you will have more success than the paid app.
"This is a pretty strange view. If you really look at our ecosystem, there are multitudes of species with a high variety of survival strategies. If there were only dandelions on this planet, the planet would be a dead wasteland. The best thing for the consumer and for the industry is to nurture a wide array of monetisation methods and game types.
"If everyone follows the same strategy, there is a high chance we will hit an evolutionary dead end and we'll all be screwed. However if we are open to new ideas there is a greater likelihood that we will hit the next big thing. Because the next big thing will end up seeming counter-intuitive until it surprises everyone."
Down to brass tacks
There were some developers prepared to argue against this however.
Thomas Nielsen from Progressive Media saw a focus on metrics and hence money as a sign of industry maturity. "Ah, if only quality was any guarantee for success, we'd all be stinking rich wouldn't we?" he questioned.
"I would go as far as saying money should always be the driver for decisions. That doesn't mean you in any way should compromise production quality, user experience, gameplay, etc.
"Newsflash: self-publishing doesn't exist. In order to be commercially successful with games, 'publishing activities' need to be performed. If you can do without a publisher, then that's because you are able to take on those roles yourself - not because the need for clever monetisation and metrics-based decision making has gone away. "
So two camps emerged; those who believed that metrics and monetisation were evil and should never be used; and those who posited a mixed model.
Joony Koo of mSonar, for example, thought metrics were fine - but they just couldn't be used in design.
"You can't get data without usage. Data is something you collect after a game is launched. In the end, metrics is a supplement to designing and improving your game. Metrics CANNOT be a game designer on its own."
Make your own decisions
He added, "I do agree that greatness is about creativeness and a pursue for originality. But that doesn't mean a game with great creativity and originality must not use data and metrics to make the game even better. It's the people who make the game who decide to use tools - not the tools themselves."
Adam Telfer of XMG Studio seemed to disagree, arguing that; "Great creative teams will use analytics as a way to test out their designs and build a more iterative process for games ... release a game early and often ... real user feedback instead of just gut intuition. Each cycle we can design, prototype, produce, test, release - all in weeks."
He continued, "I can't lie and say money isn't a part of our decision making process. Our iterative process means we have to be tight on features. We stay focused on features that grow our engaged userbase and generate more revenue. As we add new content, we have to find ways to pay for it."
Despite utilising metrics for design iteration, he didn't let monetisation affect his basic game design.
"We just launched a RPG/dungeon crawler/TD game called Powder Monkeys. Its been getting great reviews, but the monetisation isn't as high as some of our other franchises yet.
"My friend from the freemium space was quick to say. 'You have to add an energy bar and put your game 100 percent on the server. It will make you more money'. From a purist game designer stance, I would have slapped him in the face, yet I'm sure that if I added this feature our profits would surge higher then any other game in our roster."
New way works
Similarly, he pointed to the user experience design side, which is now entirely metric-driven. "The UX industry standard now is using as many metrics (and user testing) to your advantage so you can build truly user-centred applications. The idea of just using your gut is amateur."
Eros Resmini of OpenFeint agreed. "Those who are stuck in the console mentality of a pure gaming experience will fade or slowly become irrelevant. Treating high quality and metrics/monetisation as mutually exclusive in games is a mistake and it's old school thinking."
Paul Farley of Tag Games drew upon his experiences of Lemmings and GTA to argue that behaviour metrics aren't inherently bad, it depends on the user's motives.
"Lemmings spawned around ten new original ideas. Out of those ten only one was a success - Grand Theft Auto." Metrics, he argued, would have prevented developer DMA just gambling.
Elegance
Taking a historical example, Farley said, "The French architect Le Corbusier once said that 'God is in the details'.
"In my opinion, if you get those details right players will pay for the experience you are providing them with - you don't need to exploit, manipulate or build contrived systems to get there, but naturally some will. I know for us, the taste of success will be so much sweeter knowing we haven't screwed anyone over on the way to the top."
He also made an interesting prediction of the emergence of industry-specific data analysts. "Metrics, like any other tool, needs a human operator. We'll see a great many more data analysts in this industry in the years to come.
"The data analyst is not some sort of commercial saviour but should be a key member of the team spanning core disciplines of design, marketing and business intelligence."
Will Luton of Mobile Pie compared the success of its first game, launched without metrics, against its more successful second.
"...We have the qualitative as well as quantitative data. We read the reviews and forums. We see what people request and from that we build a picture. If you're already doing that, you're doing analytics.
"We also jump on to the quantitative stuff. We look at what level people drop off at, then we look at the design and see why. That's what data brings. We dig in to it and the things our players do tell us where we've made a mistake. We go in, rectify it and players get a better game.
"I don't say this flippantly, but I love spreadsheets. I like knowing exactly what people are doing with the things I've built - it makes me better."
Importance of failure
Always welcoming all new business models and platforms, Christopher Kassulke of HandyGames wouldn't rule metrics out, but argued that diversification and VCs gave a developer a strong financial base for experimenting.
"Great games and outstanding reviews alone doesn't pay the bills. I hear so often in meetings that you cannot monetise Android, iOS, Java, free, mobile games in total, and so on.
"It's strange, then, that a lot of companies do it. Try again. Fail again. Fail better. (Samuel Beckett). That's only possible if your company's generating a strong revenue stream on other products (or has enough VC money). So spread your risk and try out new things."
Revolver PR's Brian Baglow wasn't for or against metrics, but he did recognise that metric-driven companies weren't typically comprised of old-school developers.
"OK, the metrics driven companies will soak up users. They'll siphon off cash and lose their players eventually, because the experience won't necessarily be fun, or meaningful, or interesting. Just compelling.
"The new metrics driven companies are here because it's an explosive growth area. I doubt they'll find it compelling in the long term so: find out what these cynical player-gouging swine are doing right; steal it; make it good."
To wrap everything up, the person who started the argument, Graeme Devine, GRL Games popped in to, look at the effects of his rant.
"I see metrics as finding the common easy path. Finding the happy medium, this is especially obvious in fall TV shows, with network executives wondering why no one watches their blandly written, will-offend-no-one show," he argued.
"I do not see the mobile game industry maturing. This is all a sign of impending doom in my mind. Like fall TV shows, our audiences are more intelligent than we give them credit for. Or at least I think they are. I admit, I don't have the data to back that up."
