"Is there such a thing as a KPI for fun?" asked Sebastian Kriese, corporate development lead at casual gaming specialist Wooga, during his talk at Game Connection Europe in Paris.
"If you're testing a prototype for a new game, can you actually measure whether people are enjoying it after just a few days?"
The answer, as you might expect, is not really, though Wooga does have a refined process for deciding which games go into production and which are dropped into the recycle bin.
And for a studio like Wooga which, according to Kriese, wants to release hit after hit after hit, each generating millions in revenue ensuring you can follow up one success with another is crucial.
Hit or miss
"According to our estimates, the value of the App Store market in the last 12 months is $12 billion," added Kriese, "but the problem is, it's all hit driven.
"Our estimates suggest less than 0.1 percent of games on the App Store are ever what we would call successful."
Essentially, it's Wooga's mantra that, without a hit behind you, you're nothing. You're a nobody. As a result, at any one time the company has 60 prototypes in development.
Prototypes go into user testing immediately, Kriese revealed, with people picked up off the street many who have never played games before to take on said titles and give feedback.
"Sometimes it's good to give those people prototypes you suspect aren't really working, if only so you can compare their reactions to the prototypes you think do work," added Kriese.
"Then the next stage is that the game goes into a 'gate meeting'."
These meetings involve a simple thumbs up, thumbs down system, where the company decides very quickly whether a prototype should be sent into production. If the answer is 'yes', then the game is shared with Wooga's entire employee base complete with analytics - once a playable build is available.
In for the kill
"Of the 60 prototypes we have going, only 15 will ever make it into production, and only 10 of those will ever be soft launched," said Kriese.
Wooga's soft launch territory of choice is Canada: "It's English speaking, it's big enough, and it gives a good indication of the US market as well," he added.
"Everybody does it. Those poor Canadians! They must think all games are crap."
Wooga's latest hit Jelly Splash
Depending on the results of the soft launch, only 5 games from those 10 are ever launched, and the company believes around 2 of those will go on to become certified hits.
"At Wooga, we celebrate killing games," concluded Kriese.
"We think it's so much better to stop games that are not successful, that are average we only want to release hit games."