Feature

How They Turn the Screw: A Data-Driven Approach to Ranking F2P Monetisation Efficiencies

Down, dirty and aggressive with zip gun analysis

How They Turn the Screw: A Data-Driven Approach to Ranking F2P Monetisation Efficiencies
This article was created for a talk at the Evolve day as part of the Develop in Brighton 2013 conference.

Its inspiration came from the way many people in the mobile games industry talk about 'aggressive monetisation'.

Like art and pornography, everyone seems to know 'aggressive monetisation' when they experience it, but this is a purely subjective viewport.

As I have heard anecdotally many times before, one player's 'aggressive monetisation' triggers another developer's cry concerning 'Look at all the money that game is leaving on the table'.

So being a fuzzy numbers sort-of-guy, I pondered whether you could come up with a process that would rank each game in terms of its monetisation efficiency?

(Personally, I don't like the term 'aggressive' as it's currently used with a strong moral component that is functionally meaningless. If we're going to be moralistic about it, at least let's stick some fuzzy numbers up the flagpole.)

So, in that context, this talk is my attempt to come up with such a quick and dirty process to solve this issue. Even if I don't, I hope the process will illuminate some of the factors that underpin 'aggressive monetisation' and allow us to talk about it in a more nuanced and informed manner.

1. How They Turn the Screw: A Data-Driven Approach to Ranking F2P Monetisation Efficiencies

This is the title of my talk. 'How They Turn the Screw' was added for effect, the other part of the title being considered somewhat dry.

2. Download the Develop in Brighton app - score my talk

In keeping with the extremism of this talk, the scores I like are minimum or maximum. I really don't care which.

3. Who me?

Editor-at-large at PocketGamer, the leading website for mobile gaming, both on a consumer and business level.

Tweet to @pgbiz for questions, clarification, sarky comments etc

And we have one of our Mobile Mixer events tonight at 7pm at Coalition, 171 King's Road, 5 minutes away. Free booze etc...

4. How They Turn the Screw: A Data-Driven Approach to Ranking F2P Monetisation Efficiencies

This talk's boilerplate states: 'The free-to-play business model is taking over gaming, especially in mobile games. In many circles it remains controversial in terms of how many companies employ it, yet apart from a game's position in the top grossing chart, there is no general way of ranking these games.

This talk will detail an attempt to create a data-driven process that enables anyone to generate a simple graph-based system to rank free-to-play mobile games in terms of their monetisation approach and their commercial success.

The process uses only open sourced data and is designed to generate strong conclusions (both positive and negative) about how efficiently and successfully specific free-to-play mobile games operate.'

5. A plan, a cunning plan

There are plenty of services which offer free and paid-for analysis about the free-to-play mobile games market: for example App Annie and Distimo focus on app store data. Over time, and cross-checking real data with their clients, they've build models of how many downloads, or revenue each position in a country chart requires.

(Although games such as Clash of Clan, Puzzle & Dragons etc cause problems because they can't be modelled by such retrospective methods.)

There are also companies like EEDAR (Electronic Entertainment Design and Research), who will produce detailed research on how games work.

But these cost a lot of money, or take a lot of time and expertise to attempt to do yourself. There are also a bunch of academics and consultants looking at this sort of thing in terms of retail psychology and game design.

I am not doing any of these things. I am doing something quick and dirty. It might work, it might not...

6. Some inputs

Every process needs inputs, but you can't have any old process and any old inputs. The two need to be matched in some manner. I'm starting off with -


  • 6.1 As simple and as few inputs as possible

  • 6.2 Sourced from open data - App Annie etc

  • 6.3 A quick process (minutes not hours per game)

  • 6.4 Generate conclusive results (positive and negative)


7. Some outputs

I like graphs, so my original thinking was could I produce a cross graph of commercial success (horizontal axis), versus some sort of Monetisation coefficient.

(Note: this graph is technically wrong (as the central point isn't (0,0)), but cross graphs are more interesting than ordinary graphs and I'll fudge the results anyhow.)

More importantly, as I'm doing something that's quick and dirty, so I don't really want too much granularity in the middle of the graph. Or, at best, I think I want a limited set of coordinates.

More likely, I want a compressed graph because I don't care much about games with average commercial success and monetisation. I only care about the extremes of each axis i.e the size of the yellow 'X' is the number of titles in each area

We're on a hunt for the outliers - in terms of games which have been very successful and have a high monetisation coefficient and those that have a very high monetisation coefficient and haven't been successful.

8. What is this monetisation coefficient of which you speak?

Hold your horses. That's what our quick and dirty process is all about...

9. To the process...

Let's take a selection of games and play them for five minutes. (This is the quick part)

What obvious information can we generate?

9.1 General stuff

Name, developer, publisher, free/paid, age rating, (platform - but I'm only doing iOS)

This is boring but very useful in terms of providing filters - in terms of publishers, age rating, paid/free2play

9.2 Commercial success

Unlike App Annie and Distimo, I'm not going to spend my time reverse-engineering the revenue value of each country top grossing position to create an overall revenue total. If you want that, you can pay for it.

So I'm going to have to come up with a Success coefficient as well as a Monetisation coefficient (I'm using the term coefficient in a very quick and dirty manner).

In this process, I'm going to record...


  • The number of countries in which a game has been a top 10 top grossing game,

  • The number of countries in which a game has been a top 100 top grossing game,

  • The peak top grossing position in the US


Playing around with numbers, I come up with something I call the 'Success coefficient'.

So, let's take the number of countries in which a game has been top grossing top 10, and divide it by the number of top 100 chart peaks the game has had (this gives us a strong filter for success: and a number from 0 to 1). Of course, you could do top 5/top 10 to further sharpen this (or top 10/top 1000 to blur it) but this is our initial process to create some sort of process.

Then we'll strength the filter by dividing the previous number by the game's peak US top grossing position. (Obviously you can use other countries - China, Japan, Korea - if you want).

And neatly, our Success coefficient will run from 0 to 1, with 1 being awesomely successful and 0 being a complete disaster.

The results look - okay


  • Blood Brothers - 0.06

  • Jetpack Joyride - 0.10

  • Megapolis - 0.19

  • Marvel: War of Heroes - 0.22

  • Hay Day - 0.49

  • Rage of Bahamut - 0.74

  • The Simpsons: Tapped Out - 0.87

  • Modern War - 0.93

  • Dragonvale - 0.95

  • Candy Crush Saga - 0.97


9.3 Monetisation efficiency (ahem, aggressiveness)

So what inputs can we generate for monetisation efficiency? Note - we're measuring time-dependent inputs during the first five minutes of play.

9.3.1 - Minimum and maximum IAP prices, and the number of IAP bands

9.3.2 - Time to Store (this likely needs to be better defined in terms of hard, soft currency)

Some examples of Time to (hard currency) Store


  • Frontline Commando - 4 minutes

  • The Simpsons: Tapped Out - 4 minutes

  • Hello Kitty Carnival - 3 minutes

  • Frontline Commando: D-Day - 2.5 minutes

  • Megapolis - 1.5 minutes


9.3.3 - Number of store visits in 5 minutes

9.3.4 - Alternative currency sources (ads, offerwalls, gifting, daily bonus)

9.3.5 - Number of hard and soft currencies, and number of resources (recorded in our 'Confusion coefficient')

9.3.6 - Any warning given about IAPs?

Looking at these input, we can build an equation that seems to work across all games.

(min IAP * max IAP)

This gives (99c * $99.99) as a default benchmark

Then we'll use Time To (Hard) Store as a divisor, and multiple by our 'Currency Confusion coefficient' (the number of currency and resources types in the game).

The result is = ((min IAP * max IAP)/Time to Hard Store)*Currency Confusion

Examples:


  • Jetpack Joyride - 6

  • Rage of Bahamut, Marvel: War of Heroes, GI Joe (all Mobage games) - 20

  • Hay Day - 20

  • The Simpsons: Tapped Out - 49

  • Frontline Commando - 125

  • Dragon Eternity - 150

  • Megapolis - 247

  • War of Nations -249

  • Frontline Commando: D-Day - 333

  • Modern War - 499


To 'coefficient it', this number needs to be divided by something [(99c*$99.99/10)*2) i.e. 20]. We come back to this later when considering a standard for the horizontal axis.

(In this vein, a 'maximum value' for the equation would be (500/1)*6 = 3,000)

9.4 Our process morphs into something else

Playing games for the first five minutes is an excellent example of the power of quick and dirty, because doing this we also get the opportunity to review the initial user experience, which isn't a hard metric like success or monetisation, but is still useful. For example we can generate the following metrics...


  • 9.4.1 - Time to load

  •  9.4.2 - Time to Push notifications

  •  9.4.3 - Time to Social

  •  9.4.4 - Frequency of share prompts

  •  9.4.5 - Time to Failure (best example is CSR Racing- although not in this survey)


10. Notable examples

  • Hay Day (0.48, 20) - First thing you see is an IAP warning pop up. Only get the push notification option the second time you log in

  • Dragon Eternity (0.003, 150) - Start playing in your underwear; an encouragement to buy clothes.

  • Megapolis (0.19, 247) - Get the option to spend $99 within 5 minutes. Was the only game to prompt iTunes rating in 5 minutes.

  • Dragon City (0.03, 179) - Only get push notification after you've finished the tutorial.

  • Frontline Commando (0.02, 125) - Don't get to see real money value in the store, only in the iTunes prompt.

  • Mobage - All games have a hard gate wrt social log in, and all game have a Monetisation coefficient of 20.

  • Ice Age Village (0.08, 40) - Very weird/random note to parents about IAP.

  • The Simpsons: Tapped Out (0.87, 49) - First thing you experience is a 283 MB update, then a very funny, unskippable cinematic. Offer to hard currency within 5 minutes.

  • Hello Kitty Carnival (0.002, 10) - First thing you see is Facebook Connect prompt, and you get hard currency prompt in 5 minutes

  • CSR Racing - Time To Failure - 2 minutes (neat way to get you into the store)


11. Making graphs

Using the same information, how can I arrange the graph axis to best display me results? I'm think you use non-linear scales for each of the quadrants to highlight the numbers you think are interesting.

12. General conclusions


  • The cheapest IAPs are now often $2.99 or even $4.99, not 99c.

  • Time-based games - generally easier to get you into a shop or store environment.

  • Only one game in my survey contained banner ads (likely more on Android).


13. Limitations of my quick and dirty system


  • I was only looking at iOS but even then you still have to deal with separate iPhone and iPad apps for some games, which causes confusion.

  • This process is US-focused (issues with Japanese, Korean App Store success).

  • I need to better define Time To Store in terms of the soft/hard currency prompts and the granularity therein.

  • This process doesn't really deal with the split between Time-based versus Skill-based games. Can we come up with a better classification? Probably difficult within a 'minutes, not hours' timeframe.

  • This process heavily focused on early overly aggressive monetisation than changes which happen in gameplay around the hour-mark i.e. it's more about retention than the sort of monetisation that drives the most successful games.

  • Is the crossed M coefficient predictive or created to fulfill my gut feeling? Many, many more games required to be analysed to decouple the process from the small data set.


14. For further investigation


  • Is Time To Finish Tutorial (TTFT) a useful metric?

  • How does this process handle time, esp. in terms of the Success coefficient? Do we need - say - a 2 months from launch work up period?

  • Can we do enough research to look at company philosophy (ie Glu, Zynga vs Supercell, NimbleBit, Mobage)?

  • Can we look more deeply at the first store experience in terms of how soft and hard currency is offered?

  • Can we create a social coefficient, then combine it with the monetisation coefficient as virality is becoming a much more important factor?


NB: In keeping with my quick and dirty philosophy, the basis data analysis for this talk was carried in over two days, with analysis of each of the fairly randomly selected 40 games taking around 30 minutes. The point of the process wasn't to be precise, but to create something interesting that could be expanded upon in future (if required).

However, I'm very happy with the results, and encourage readers to get quick and dirty themselves and come up with their own zip gun processes of analysis.



Contributing Editor

A Pocket Gamer co-founder, Jon is Contributing Editor at PG.biz which means he acts like a slightly confused uncle who's forgotten where he's left his glasses. As well as letters and cameras, he likes imaginary numbers and legumes.

Comments

No comments
View options
  • Order by latest to oldest
  • Order by oldest to latest
  • Show all replies