Analytics & Data

9 steps to using analytics in your game

How to harness the data for your game design

9 steps to using analytics in your game

Oscar Clark is an evangelist at Unity Technologies and co-founder at Rocket Lolly Games.

This article is part of the Mobile Games University: Guide to Game Analytics 101.

You can also check out our Mobile Games University track at Pocket Gamer Connects Helsinki 2017 on September 19th to 20th to learn more about the games business.

The realm of data analytics is daunting for the uninitiated. The language used so casually by some is meaningless jargon to too many of us and can be disempowering.

As a result we can too easily give up and either end up doing nothing or paying some “Data Genius” to magically solve all our problems. It rarely works like that.

We need to know why a player decides to install our game, what triggers their initial play. What causes them to churn out of the tutorial or continue playing to the next level, why they return the next day… the list goes on.

To do that we need to understand the journey our players embark upon when they start our game and look at how we can implement changes that are likely to have the most impact. That requires Insight; something we only get if our data is structured in a way which mirrors the game itself.

These nine steps will help you get your data into a shape where you can understand what’s going on and make educated guesses about how to improve the performance of the game.

Click the link below to begin.


Click here to view the list »
  • 1 Set objectives

    Despite the way some people talk, collecting data is not an objective in itself. If we want to make informed decisions we need to start with a set of objectives.

    We need to understand our goals if we are to understand whether we are collecting everything that is relevant. Some people do try to capture everything and hope that they can work out the rest later; but this comes at a risk that we will bestowing random data points with apparent meaning with the gift of hindsight.

    Capturing the right data

    Knowing what you need to find out is half the battle when it comes to working with analytics. Look at your game as a hypothesis. An idea which requires proof. It matters if you are looking to “find the fun”, how to maximise monetisation or to calculate LTV (life time value) so you can work out if it’s worth spending money on user acquisition. These decisions affect how you design the capture of that information.

    Most projects have multiple objectives, each of which you need to try to describe, but there will be priorities as the way we capture data has a significance in how we interpret it.

    Common goals include:

    • How far will the player progress in the game?
    • How long can I retain the player?
    • Can I attribute where this player comes from?
    • How effective is the monetisation? (On D2, D7, D15, D30 users)
    • If players complete the tutorial does that impact retention?
    • What is the life time value of the player? (Over 30 days? Over 90 days?)
    • How engaged are players? (And how does frequency of play impact LTV?)

  • 2 Define what you can measure

    Once we understand what we want, we need to look through the flow of the game and work out what we can measure. We can’t measure ‘fun’ but we can interpret ‘engagement’ by the number of times per day that the player returns.

    Looking at how many players return on day two; day seven, day 15, etc. shows us our retention. We can get insight on the value we are offering by looking at purchase events and video-ad views and their impact on player behaviour.

    We can look at frequency of paths players take through a game map and how often they find Easter Eggs.

    Take a look at the variables in the game that show you events where a player made a choice. This is the start of your data model.


  • 3 Define the trigger events

    Most Analytics tools have some basic start-up data including an anonymised PlayerID, StartTime, perhaps even DeviceID.

    Some platforms like Unity Analytics have a whole set of standard events intended to streamline the setup of standard elements from onboarding, monetisation, progression, etc.

    Always check the basic implementations for your platform of choice. This can really save a lot of time and will usually improve the results you get from that service.

    Even if you are using a fully featured platform it is likely you will still want to create a series of ‘custom events’ which allow you to try to answer questions very specific to your game design.

    Identifying triggers

    The event are specific triggers we can use need to give some deeper insight to the behaviour of that player whether that is a specific path taken, a shot, a trap triggered, a line of candies swiped.

    It’s essential to define these trigger points in a way which allows us to make useful comparisons rather than duplicate any existing standard events. Generally, this means defining the event as a moment in the flow of play, rather than being too worried about the details that will be captured at that point as data.

    So rather than creating a trigger event called ‘EndLevelOne’, we look at ‘EndLevel’ and make which level the player has completed a data point instead. So if we are looking for the players' progress we should look at the events like ‘StartSession’, ‘StartLevel’ and ‘EndLevel’ if your game has that kind of progression.

    If the game has a series of objectives you may have ‘ObjectiveComplete’ with a list of events as data. If you are using an experience points-based system you may want something like ‘XPAwarded’ with a score.

    As a quick aside its worth checking if your analytics platform supports standard terminology for the events you need to support. If you can leverage common terms for these triggers with other games it will make it much easier to gain deep intelligence about how your game performs.

    At the end of the day events allow us to capture a meaningful moment in time; a snapshot into the players progress in the game and usually specific functions in the code of the game.

    We use them to paint a picture of the player experience in your game. The following show a set of some common events that can help you get started thinking about what your game needs (n.b. these are generic examples only):

    • GameMenuLaunch
    • SessionStart
    • LevelStart
    • OptionSelected
    • StoreOpened
    • PurchaseStarted
    • PurchaseComplete
    • VideoAdStarted
    • VideoAdSComplete
    • MissionSet
    • MissionComplete
    • LootFound
    • TutorialStepStarted
    • TutorialStepComplete
    • NotificationsEnable
    • DailyRewardCollected
    • BoosterUsed
    • SocialShare
    • PlayerInteraction
    • PlayerAttacks
    • PlayerHit

  • 4 Classify your data types

    For each of our custom event there are three kinds of data point: common, reference and custom.

    Common data point: Standard markers we can use to compare events with different users to create cohorts and to track longer-term trends. As with standard events you may find a lot of these data points are pre-defined and collected so it’s important not to duplicate.

    N.B. Always make player tracking data anonymous and keep that separate from any personal data, we just need the information that affects the operation of the game. Tracking Personal data is a security risk and has legal implications in terms of data protection and duty of care.

    The kinds of common data include:

    • Date/time
    • Anonymous PlayerID
    • SessionID
    • X, Y (, Z) position in game

    Reference data point: These are game-specific elements which don’t change regularly. For example the specific level design, the build a given player was using at that time, the device a player is using.

    Similarly, we don’t need to post the specs of the weapon a player used in the game; but the ID for that weapon and the damage done. We need to confirm some of these at the start of play, but importantly they won’t change during play, so there is no need to capture it again and again.

    Where we can, it's more efficient to be able to infer as much as possible. For example, if we know the x,y,z coordinates of a player at the time they act in a game level then we don’t need to capture that they were in the trigger zone for a trap or treasure chest.

    The reference data will tell us that when we analyse the reports. Typical examples include:

    • Game level
    • Game rewards
    • Level objectives
    • Purchase SKU (stock keeping unit)
    • Player geographic data (where available)
    • Player demographics (where available)
    • Game build

    Custom data point: These are the game specific variables which we need to capture at the time of the trigger event. They describe the detail behind the status of the players related to that event and any associated references e.g. the ObjectiveID of the action that occurred or LevelID played.

    Typical Custom Data Points include:

    • Score
    • Ammo
    • Health
    • Damage
    • Time
    • LevelID
    • BoosterID
    • ObjectiveID
    • RewardID

  • 5 Implement data events in the game

    Now we have defined the events and the data points we need to put them together and set them up in the game code, so we create a set of events and define the data points which will be captured and posted to the server.

    The following table shows some examples of events and their associated data points. You could theoretically build this into a database but it’s much more efficient to use one of the games analytics tools, like Unity Analytics, which make set-up and reporting significantly easier.

  • 6 Map data to the player experience

    Collecting the data is only the first step. You have to be able to make sense of it (another reason to use an off-the-shelf games specific analytics platform).

    If you just look at raw data without understanding its structure it could easily lead you astray. I’ve seen to many developers assume that they can make decisions based on simple averages. Averages can be meaningless; there is a ‘Tyranny of Averages’ because the ‘mean’ of the data doesn’t tell you anything about the distribution of the results.

    Commonality

    We need a common starting to compare players with a common reference point. My favourite example is also perhaps the simplest. What if we compare player activity based on the number of days since they first played the game. This is surprisingly effective.

    It takes us away from the messy snapshot of all players who happened to play on the day we sampled and instead gives us a chance to compare players at a similar stage in their player lifecycle.

    We can get much more sophisticated than that by building cohorts. These are just collections of players who share common characteristics.

    For example, let’s say we take all players who are using Android 5.0 devices in English Speaking Countries; who signed up in the last four weeks. We can then compare the behaviour of this cohort in the game with a group with a different set of criteria e.g. iOS 6.0 users in non-English speaking countries or even Android 5.0 users who signed up between four to eight weeks ago!

    This is where the real insight can come. By properly assessing the trigger events and the best use of data points we can get real insight into player behaviour and compare the impact of changes we make. It allows us to test our assumptions with evidence.


  • 7 Use robust reporting tools

    Having gone to the trouble of creating a solid data model for you games it's worth investing in a reporting tool that you know works.

    Don’t rely on an Excel Spreadsheet if you want to be sure to find trends. If you are using an analytics platform like Unity’s there should be a suite of reliable reports from active players, retention to funnels and even heat maps.

    Funnel Analysis: Identify the steps in the flow of the player experience and track these for your cohorts. This gives you an early warning system to identify where players are breaking out of your experience

    Heat maps: An amazing tool to allow you to visually assess player patterns and one of the best ways to understand how players actually play your game in practice. Look at the patterns that your players use in terms of position and interactions over time and map that directly onto the level.

    KPI data: Think about the measures of what success look like in terms of retention, monetisation, acquisition, attribution and how any changes you make affect your success measurements.


  • 8 Act and record changes

    Once you have your data you need to work with it to better understand what the information means. Quite often the information you get shows just a set of simple average numbers, matched to different stages of your players.

    Understanding what is ‘good’ or ‘bad’ can be very difficult. The information we gain shows us what happened in the set of circumstances our players experienced at the time.

    Interesting; but this only becomes useful when we start to act on that information. Just because you have data points, doesn‘t mean you understand your players.

    Remember what we said about needing to understand our objectives. We need to explore the questions we set ourselves at the outset and see how small changes can dramatically impact our performance.

    Understanding the data

    Typically for me this starts with a classic funnel analysis where I need to map players against the stages in the flow of the game and see where I am losing the most players. If I look at a funnel mapped against the First Time User Experience I can very quickly see what stage seems to be causing us to lose the most players.

    We can then look at that part of the game again; think about some possible reasons – perhaps even ask some players about what they think - come up with a theory to test. We can implement that change and see if it has the effect we are looking for.

    This is way more effective if we only apply the change to a statistically relevant sub-set of your players, rather than risk problems for all your players. We can compare results between Group A and B (or groups C, D, E) to work out the best and most effective solution. Although generally it’s worth changing just one thing at a time so you can attribute the effect with some degree of safety.

    The key here is generally it’s not the base number itself that we are interested in. It’s the difference we can affect and what comparisons we can make to improve the player experience.


  • 9 Review the process itself

    When you look to set up your data platform it’s easy to assume that this is a one-off process, but in fact, just as your game needs to be reviewed, so do your analytics.

    Don’t assume that the way you are collecting data is right. This is also a hypothesis and needs testing too. Keep testing the process itself to make sure that there are no unintended biases or misreporting.

    Similarly, continue to consider the KPIs and measurements you are looking at. Are they still relevant as your company becomes more mature? Are they helping you get the most effective results for the game. The switch from game design to operations requires a switch in your mindset too.

    In Summary

    Getting your analytics right matters because it’s the best way for us to understand not just what players tell us about the game; but actually what they do!

    This is now a vital aspect of the development experience and we need to carefully map the way we collect data with the way our players flow through our game. That means leveraging the power of the tools which are readily available to us and looking for those specific elements where our game needs something extra.

    1. Set objectives
    2. Define what you can measure
    3. Define the trigger events
    4. Classify your data types
    5. Implement data events in the game
    6. Map data to the player experience
    7. Use robust reporting tools
    8. Act and record changes
    9. Review the process itself

    We don’t need to be data analysts to do this; we need to be data aware and be unafraid to follow the steps to ensure that we understand what our data means so that when we start experimenting we know what measures we are trying to change and why. And we can get better results too!

    This was part of the MGU course Mobile Games University: Guide to Game Analytics 101. For more articles in this course click here.


PocketGamer.biz regularly posts content from a variety of guest writers across the games industry. These encompass a wide range of topics and people from different backgrounds and diversities, sharing their opinion on the hottest trending topics, undiscovered gems and what the future of the business holds.