Boiled right down, games development is a series of tough decisions, whether you're ironing out that difficulty curve or, as is increasingly the case, determining your in-app purchase pricing structure.
All developers have to make big calls when it comes to the direction of their games, and for the most part such decision have to be made without the benefit of a cache of relevant data.
There are companies, of course, who are looking to offer solutions. Newly launched service called Splitforce is offering Unity, iOS and Android developers a tool that allows them to A/B test their entire app against a slightly different version of itself; an app development battle akin to the Shadow Link mini-boss fight from Ocarina of Time's Water Temple, if you will.
"Splitforce allows anyone from indie developers to product and monetisation managers at game studios to publish different versions of their product simultaneously," Zac Aghion, co founder of the Beijing based company, tells us.
"You can then track which version is driving desirable user behaviour so that they can make decisions to improve their app and their business without thinking twice."
Anything is possible
The beauty of it, according to Aghion, is that you can track changes to just about anything in game.
"After installing our native library into their app, you can setup and publish experiments on Splitforce and then track how they are affecting user behaviour," he says. "Anything you can code, you can test from UI layouts and price points, to game mechanics and IAP checkout funnels."
Best of all, developers using the service can make changes on the fly to the live version of their app.
"You can even publish new versions or delete experiments instantly, without having to recompile your app code or re-submit to the app store" - a major bonus to developers who are looking to avoid the app store submission rigmarole.
Though new on the scene, the team behind Splitforce isn't without significant experience, however. The company's founders previously worked as A/B testers in Canada back in 2009.
"At ILABS in Montreal, we pioneered advanced techniques in multivariate testing applying six sigma methodologies to landing page experiments with the effect of increasing testing efficiency by over 100x," details Aghion.
It was during this time that he learned the frustrating state of play facing mobile developers.
"I came up with the idea for an A/B testing tool for apps when I was speaking to my friends at Optimizely, a leading A/B testing service for websites. They told me that they had been getting a lot of requests A/B testing native apps, but had no such product or service. I decided to address this market need. "
This led to the foundation of Splitforce in June 2012 and it rapidly garnered interest and investment.
Backed by Chinaccelerator (as well as boasting a number of clients including EVERYDAYiPLAY, ReignDesign and Mobile Now Group) the offering has been dubbed a "game changer" Unity China's Allen Foo in part thanks to the unique advantages it offers to Unity developers.
A/B testing for dummies
In particular, Splitforce hopes to provide the simple solution to the A/B testing conundrum.
Crucially, Aghion is hoping to open the whole process up to the masses to enable developers to make better decisions a boost that will aid devs and player alike.
"A/B testing has been difficult for app developers thus far because they've needed specialists to work on both implementation and analysis," he says. "With Splitforce, none of that is needed. Setup takes just minutes and we've put a lot of focus into making our library intuitive and flexible."
It's a pitch that's likely to appeal to a solid body of developers. After all, one of the big barriers to entry that smaller outfits have been forced to suffer when chasing the freemium dream has been the lack of easy to use resources that allow a small team to optimise a game with the rigour and panache of a King or Supercell.
And while Splitforce seems to be a very practical solution, the price of the service could potentially prove a barrier to entry to those lower down the chain.
Running tests with 5,000 Daily Active Users is going to set you back a wallet damaging $299 a month - chicken feed to a big developer but likely to force indie developers into the free package for only 500 DAUs and leave them scrambling for money to pay 5c more for every user over that limit.
Nevertheless, Aghion remains bullish about the prospects for Splitforce. In his words "it helps app makers improve their key business metrics" and, as a result, looks like it could open up A/B testing to a far broader market.