Comment & Opinion

Defining mobile analytics 2.0

Adjust's Paul Müller on what comes next

Defining mobile analytics 2.0

AppInTop is an automated mobile app marketing platform that combines the mobile traffic from 30 countries into a single interface.

It also runs a regular marketing podcast, with PocketGamer.biz publishing transcripts from the most interesting discussions.

You can subscribe to the podcast via iTunes here or listen to this episode here.

In this week's article, Adjust's CTO Paul Müller talks about the company's approach to mobile analytics and what trends he expects to experience over the coming months.

On your website, you state that the mobile industry has at best provided rubbish-in rubbish-out data feedback to advertisers. What's wrong with the state of mobile analytics today?

Paul Müller: Often analytics, specifically attribution and tracking, have come from online space. So there was an attempt to apply a web-focused approach on how a user interacts with a website to how people interact with apps. That does not add up
.
We see a huge demand for cohort analysis. This is because it's the only way to make any sensible assumption on how users use the apps. However a web-focused approach doesn't show you the full picture and can yield misleading results, which in turn leads to spending money on the wrong kind of users.

Many of the analytics services coming into the app space are from online world and they don't understand the new game. We have a completely different outlook on how you should perceive the user interacting with the app. There is no such a thing as an install in the web world. You have to understand and correctly attribute more fundamental central topics in terms of user acquisition.

What metrics are important?

The app world dates back to roughly 2009, but even it has gone through some changes. In 2009, one of the most basic things was understanding how your app ranked across all the different stores because, even today (but more so back then), a good source to acquire new users was through the app stores.

One of the things we usually recommend our clients do is to track how users react to call to actions. For example, if you're using something like Appirater that prompts the user to give you a review, are they reviewing you? Do they like the app? Are they using it?

Retention is a word that gets thrown around a lot and the definition is very loose.
Paul Müller

Retention is another thing which is important and it's not just an interesting metric to establish if the user sticks around.

When you're starting out, it's very important to gain a foothold in the app store ranks as usually the beginning of your life cycle is tremendously important in determining where your app will rank in the long run. So if you have failed to make it to the top ranks, your app is pretty much destined to become a zombie app.

For Android, retention is one of the critical metrics that is calculated into the ranking of your app. Therefore, if people aren't sticking around in your app, you will not rank regardless of how many users you acquire. Understanding that metric is key to understanding how your app can improve its ranking and gain more free, organic users from within the store.

Retention is a word that gets thrown around a lot and the definition is very loose as well: it means different things to different people. It starts with a session where your user opens the app, but we can see that a couple of our competitors don't define what a session is. They just take the definition from iOS, equating a session to user opening the app.

The problem is, that's not a very good definition. Every push message, everything that maybe draws focus away from the app will add up to many sessions.

Google Analytics has a smart way to do this as it collects all the basic signals it receives and calculates sessions that have a minimum 30 minutes between sessions. That's one of the definitions that actually makes sense, but no one uses it because it was technically complicated to aggregate sessions like that.

Once you understand what a session is, then retention means how many sessions a user has in a day, how many days in a row they are using it, and how many days after the install they are still around.

We've seen some very different techniques to ensure users are engaged with the app over a longer term. For many games, it is providing a lot of content. If a user plays a very short game, the user plays it for two days and there is no need to stick around for a third day.

For dating apps, they use push notifications to tell the user that there's something new, someone they can date or someone who wants to meet them etc. This motivates them to open the app again. It depends on the vertical. If users aren't in the app, there is no way that they are going to spend money or do anything you want them to do in the app.

What is another advantage Adjust analytics offers?

I come from the point of a publisher that just wants to get the job done - to build apps that people use and engage in. Everything else is noise in my typical day as a publisher.

What I want is one product where I can see how my app is doing in the app economy as a whole. This means app store statistics. I then want to know how my attribution is actually running: how many installs, re-attributions and engagements did it acquire today? I need to have the analytics to understand what the users are actually doing. So that's one part.

The second part is understanding if your app can breakeven. You could see, for example, if your organic users are returning the lifetime value that you would need to acquire new users. To answer questions of the product guy and a marketing guy in one platform is pretty much our goal.

Finally, attribution and tracking should be completely independent. We don't sell media, because we believe that it creates a severe conflict of interest.

Who are your target clients?

When we started out we had a clear enterprise-level goal. All our direct sales are concerned with big to enterprise-level clients. In Germany, we have pretty much every large enterprise working exclusively with us.
We're obsessed with simplicity and efficiency and so we built a service that is easy to use.

Then, as an experiment, we opened a sale sign-up for a long tail and ever since then we have had hundreds of smaller clients signing up for the product without ever getting in touch with us. Everything is automated.

I come from the point of a publisher that just wants to get the job done.
Paul Müller

The big enterprise clients were really good to help us figure out how to write the documentation that answers all the questions. You don't have a huge knowledge base where users have to search through the questions. That is already flawed approach.

You have to go back to the interface and change it so that a user can work with it, without looking it up. This is how we built our product and it is now used by hundreds of small clients who use the product without needing our involvement.

A small-time developer is just starting out. Is it worth it for them to start off with Adjust or are free basic analytics services sufficient?

Just today, I had a couple of very interesting talks with some more experienced app developers who have split off from the big traditional app publishers and are starting their own shop and the first thing they've put in is tracking.

Not even test versions are released without proper tracking because they understand that F2P is a metric-based game. If you don't know the KPIs of your app from day one, you may as well not try. Of course there are the Flappy Birds of the world and everyone wants to win the app lottery by just building that one smash hit, but for the rest of them it's hard work and it has to be measurable.

You have to start tracking the earliest version of your app to see whether the new versions have improved your KPIs. You have to choose your KPIs and optimise your version to match those KPIs then release the app on a bigger market and see how that's doing.

Then you start doing marketing based on those KPIs. You also have to constantly monitor lifetime values, ARPU, ad dollars, conversion rates for certain events, virality of your product. These things are important from day one regardless of how big or small you are.

How can you help prevent fraud?

I wish there was an easy answer to that. We have a lot of networks and partners approaching us trying to work with us to battle fraud. First of all it is their problem because when they're selling fraud they get an image they don't want.

It starts with simple things. For example, our SDK runs a slightly more complex code in the app, so it is slightly harder to fake it, so you have to simulate the app.

It may sound simple and we see a lot of simulated installs, but for many other tracking providers, you can just manipulate the HTTP API and just use that and it costs you nothing. We would easily recognise a non-real device that's trying to fake the installs from the SDK.

Another thing is that due to the things we actually do in the app and what we track, we have a lot of possibilities we can check. It starts with simple things like checking against the IP. Does it come from known data centres or cloud? Is it AWS? Google App Engine? Is it Microsoft Azure?

We also enable our clients to do more complex things too like tracking scammers who simulate installs on some kind of data centre with proxies and VPNs to tunnel out into a target country. They usually don't think about the little things, just to give a small example but not give away all the tricks, they usually forget to set the local clock. So for example so a German device shows a Chinese time zone. That never occurs in the wild. All Android devices are set automatically, and this is one of many simple checks that many fraudsters fail.

It isn't possible to completely prevent fraud unfortunately. It is a cat-and-mouse game. What you can do is to make it a lot harder for fraudsters and not profitable anymore.

A lot of publishers have their own affiliate networks. They're looking at is retention because that's something that's very hard to fake as you would have to assimilate tens of thousands of devices for days and that becomes unprofitable because you only pay so much to have a certain event happening. They check what the three-day retention or seven-day retention is and that is usually zero for faked installs.

What are the most important features that your users need on a daily basis?

There are basically two sets of 'features' we provide. We look at the problem from app developer's point of view.

Being a German company, we adhere to all the German privacy laws. This means that we have to be able to tell you where the data is stored, what we actually store, how do we store it, who has access to it. How does it go from the SDK to the server?

We only and exclusively use our own banking grade SSL encryption for that. Astonishingly, this is not standard in the industry. Most data from apps is actually sent clear text to tracking partners which I personally find completely unacceptable. The reason we built in such things is to be able to forward and share the data with the client in real time with their own BI system.

Developers are very pleased that we simplify things like campaign setup. For example, if you wanted to run a Facebook re-engagement campaign it would take about three clicks. For our competitors, that takes ten times as long.

The same goes for open-source SDK. If it complies with your app, you don't have to worry about what platform it will crash on because when you get the compiled library, there's a good chance it hasn't been compiled with the same settings. It may crash on old iPods. I've been there, I have had hundreds of angry reviews because SDK was crashing on some old iPods.

It wouldn't happen with the open-source SDK because you get to compile it. You also get to see what it does so if there's a problem you can directly tell us if there is something weird in the SDK, e.g. 'In line 25 you guys made a stupid mistake'. Sometimes they can just go on to GitHub where our SDK is hosted make a pull request and make suggestion how to make SDK 'nicer'. We can incorporate that into the next version of the SDK.

This is much more efficient for the clients to be able to interact directly with our developers through GitHub. It all adds up to the package we offer to the publisher to help them get the job done and get it out of the way.

How do you calculate the key metrics as return rate and lifetime value?

My team came from more old school backgrounds, so it's not all in the cloud. Postgres with C-functions is more the kind of thing we do and we've built a very sophisticated cohort analysis based on that technology.

We group all the users into the segments, by traffic sources, country, device. We continuously measure and update the metrics of each cohort, so we have hundreds of dimensions we can slice and dice to analyze cohorts to be able to answer the questions from the dashboard in real time.

This technology is used to calculate all kinds of metrics that are usually derivatives of absolute numbers such as lifetime value, which is derived by calculating the total revenue created by that cohort divided by the total number of users in that cohort.

We track absolute counters segmented so that we collect millions of different counters every day and then we have a very specific logic to calculate all those metrics from those numbers.

The main purpose of the cohort analysis is to tell you, which channel works best. Does a Facebook campaign do better than your Chartboost campaign? Which of your Facebook ad groups work best? Which of your Millennial campaigns perform best? Which generated the most engaged users?

As we go forward, we will also enable you to do things such as understand how different app versions work and measure that.

We were incredibly lucky to get some very experienced people from big companies to work with us. They told us, 'this cohort analysis would be perfect if I could define whatever I want in terms of grouping, segmenting and filtering KPIs'. It was a complicated design but we did it, and now it is done and it is a part of our product and it pays off with our new clients. They do not have to ask us to give them specific KPIs.

Can you evaluate the virality metric and if so, how?

The virality is the number of users each user generates for you and what makes this interesting, it consists of two parts. First of all, how many times does the user share it? This is an event that you can track this in your application. If the app suggests posting to Facebook or Instagram or Twitter and if the user agrees to do that, you should track it.

The second thing you have to do is to understand the users who are clicking on that tweet, how many of them are installing and using the app. You can do that from our dashboard. You can simply tweak one of our URLs and attach, which user shared that.

Then in our dashboard you can see not just how many times the user tweets, but how many times they tweet or post on average in the first, second, third week and so on. You can also see how many users you gained with this feature and how many of them came from that particular user.

Therefore, you can see who your most influential users are and perhaps you would like to reward them because we can put this information back into the app via the SDK. We call this 'In-App Data Access' and you can use that to incentivize users. You can say, "Hey, there's like a hundred new users because of you, so thank you. Here's, let's say, a hundred gold coins or something."

How do you monitor mobile app campaigns and TV ads?

We're currently working with another start-up and they measure TV advertising in real time using image recognition software. They're willing to share this data with us. We're testing the hypothesis that there is a correlation between an ad being shown and the number of organic installs rising. As far as we can tell, there's a very good connection in a short defined timeframe that first tests show.

You'll be able to track TV campaigns, but of course, you have to use the service so we can get your API data. We monitor a normal base, so for 5 minutes before the TV ad running you get 10 organic installs. Then suddenly when the TV ad is being shown, there is a huge spike in the number of organic installs, sometimes thousands of installs in these five minutes. So we can safely say that those thousands of installs come from the installs that has just been shown. It's not precise, but it's pretty close.

You can then do standard attribution for this user and all the analytics. You can get Facebook or whatever partner and that would make your TV ad memorable.

There's still the intangible effect of the long term effect of TV ads, specifically if you have bigger brand names. We can see with dating or delivery services that it has a very sharp defined peak right through the time of the ad spot and the short time after it.

Losses on the tracker side can be between 20 and 50%. Is there a solution?

Yes and this problem segments into two scenarios. One of them is that we can see specifically in Android applications. The user downloads the application or the store counts the download, they may even open it, but they don't give it the proper rights such as access to the internet. This happens a lot with apps that have, shall we say, big demands in terms of what they want to be able to do on your phone and if the user just rejects it, you don't know why the user won't turn on the app.

That's one of the things we will be able to see once we get the store reports which is something we're working on. We'll then be able to give the client an alert when we see a very high discrepancy between the number of downloads the store reports and the number of installs we can actually see.

The second thing, which is a bigger problem percentage-wise and something that we figured out in our second SDK version is that the state of being online on a mobile device is far more fragile than one would actually assume.

Our first SDK had a very naïve strategy of trying to connect to the server. It basically just tried to build an HTTP connection. If it failed, it did one retry and waited a little bit. It would time out after one or two minutes and we got a lot of complaints from users saying "There's too little data, if it's coming at all."

I think analytics will move towards a platform approach.
Paul Müller

We built an offline capability into the SDK, so if something happens in the session or an event etc, it writes it into an offline queue and then it has a separate service that tries to connect to our server with the logarithmic backup logic. At first it tries every couple of seconds and then longer periods and when it finally reaches the server, it transmits everything that has happened since it was last on the server last and the server records it. It is then removed from the queue.

This little change in the SDK led to tremendous increase in terms of what we could see in sessions and events for some of our publishing clients like magazine publishers and we worked together with the biggest newspaper publishers in Germany. We could see the increase of sessions and events was up to 30%, which means 30% of the time the users were not online.

If the mobile device is not online it doesn't necessarily mean that they have turned off the device or put it in flight mode, it could just be that they switched between 3G and Edge, which can lead to data loss. If they even switch from 3G to wi-fi which interestingly enough also leads to short gaps, it will just drop the connection. We worked out that once you remove the component of the tracking being dependent on the internet connection, you can cover most of those gaps.

What do you think is the number one biggest change that will be happening in the near future for mobile analytics?

I think the biggest problem is one I mentioned earlier; there's still a lot of fragmentation. You have all kinds of services trying to solve various specialised problems, so the publisher has to log in to 5 different dashboards just trying to make sense of the data. Often the real story behind the data gets lost due to the fact they all of these services aren't sharing the data with each other.

I think analytics will move towards a platform approach, so there's one basic source for the data. There may be specialised providers on top of that too but there is one platform that gives a big picture: this is the reports coming from the store, this is the data we have collected from your app and this is the story in between. So the next big thing is the service that will provide a lean comprehensive overview of all of the aspects of your app in a quick and efficient manner and gives you more time to focus on your app.

This is an abridged and edited interview from the AppInTop mobile app marketing podcast produced by AppInTop, an automated mobile app marketing platform.

Listen to this podcast episode or subscribe to the podcast via iTunes.


Tags:

PocketGamer.biz regularly posts content from a variety of guest writers across the games industry. These encompass a wide range of topics and people from different backgrounds and diversities, sharing their opinion on the hottest trending topics, undiscovered gems and what the future of the business holds.