Comment & Opinion

Inside the Game: Small team, lots of devices to support

Henry Shao on QA challenges

Inside the Game: Small team, lots of devices to support has partnered up with US developer Pixelberry Studios to highlight its candid stories on the trials and triumphs of a startup game studio whose debut title High School Story stayed in the top grossing top 100 chart for a year.

This bi-weekly series of articles will provide a mix of drama, detailed learnings, and actual numbers from their experience launching and supporting a top game.

Henry Shao is the QA Manager at Pixelberry Studios.

Back in 2010-2012, I was a Quality Assurance (QA) Manager for a small studio that made games for mobile devices within Electronic Arts (EA).

The experience I had at EA was quite enjoyable, especially when it came to the wealth of resources I had at my ready disposal. Being able to work on indie games with the budget of a large studio was the ultimate winning combo of game testing!

One clear example of this was in our device testing coverage. In order to meet all the hardware and software requirements, our games needed to be compatible across the top mobile platforms.

Pixelberry's debut game High School Story

Not only was my small team understaffed, I did not have all the required mobile devices to accomplish these goals. Fortunately, this was where the deep pockets of a large company were much appreciated.

First, I had the assistance of QA teams from different branches of EA available to me. Also, I was able to utilize EA’s funds to purchase any device I wanted, with just a simple phone call or email to the procurement team!

Not having to worry about resources and hardware made the testing process go much more smoothly!

Enter startup: making adjustments to save time and money

As we know, the gaming industry can be very cyclical, and eventually, EA decided to lay off everyone from our small studio. A month later, I joined Pixelberry Studios.

Being the only QA person at Pixelberry, I had to establish a team from the ground up and ensure maximum device testing coverage for our first game, High School Story.

By paring down the list of devices from 8,000 to around 40, we were able to save a great deal of money.

Essentially, our testing requirements were similar to those at EA, perhaps even higher due to everything riding on one game, only with a much more scaled down budget!

I quickly got to work, developing a plan to save time and money, while still trying to ensure a high quality product.

Step 1: Establish Which Platforms to Support

iOS and Android are by far the most popular mobile operating systems and received most of our focus. Thus, Apple and Google products were high priority on our device wishlist. However, after some additional research, we also decided to support the Amazon Kindle Fire, as it didn’t require much more additional work (Kindle Fires run on a customized version of Android) and supporting it would give us another sales point and more game visibility.

Step 2: Create a Device Support Spreadsheet

With the target platforms established, I created a spreadsheet to track all the supported devices. (We here at Pixelberry love our spreadsheets!) The spreadsheet was a good reference point in cataloguing our expansive device list, and for helping me not lose my mind!

Step 3: Organize Devices into Different Buckets

The supported devices for Apple and Amazon products were relatively small and easy to identify, compared to Android. To look up the device specs for over eight thousand Android devices would take months! Instead of wasting precious time on this, the production team felt that the focus should be on the top selling Android devices in the market during the years of 2011-2012, when High School Story was in production. We wanted to make sure that we put out the best playing experience for players on the latest trend of technology. This cut the number of Android devices that we needed to research down to around sixty, a number that was a lot more reasonable to handle.

Now that we had established the key devices to target, it was time to find the device specifications. We mainly looked for CPUs, GPUs, RAMs, screen resolutions, and the lowest/highest firmware that the device can support. Once all the data had been parsed, we were then able to sort the devices into different buckets.

Sample of Android device list with specs

For example, we put all devices that had the screen resolution of 480x320 into one bucket and devices with 720x1280 into another. Within each bucket, we sorted by CPUs, GPUs, and RAMs to deduce the high- and low-end devices.

Step 4: Finalize the List of Devices to Purchase

We purchased devices from each of Apple's and Amazon’s product lines for testing. For Android, we highlighted the highest- and lowest-end devices in each bucket from the device list, along with the top selling ones - these devices would be the ones we needed to purchase. By targeting the extremes cases, I was confident that any devices in between would run our game just fine.

Device purchasing

By paring down the list of devices from 8,000 to around 40, we were able to save a great deal of money. However, obtaining the target devices was not as easy. There were some old devices that I couldn’t get my hands on, so I scoured the internet for the next closest device with similar specifications.

Part of our device collection

For newer devices, we sent our interns out to the stores, which sometimes resulted in them having to wait hours in line for hot items!

High-end devices could display around 40 characters on the screen at a time, but on low-end devices, 8 was the limit.

Lastly, we had some Pixelberry staff who sold their old devices to the company or allowed QA to use their personal devices for testing. It was definitely a team effort!

Testing time!

When it came to testing High School Story, we ran the low- and high-end devices first, to get a gauge on game performance, especially where frame rates and loading times were concerned. For example, we found that for high-end devices, they could display around 40 characters on the screen at a time, but on low-end devices, eight characters was the limit without sacrificing game performance.

The QA team worked in close proximity to the production team. Thus, whenever a problem came up with the game, a member on the staff was able to bring it up immediately, resulting in quicker resolutions and time saving.

Results: submission and success!

In January 2013, High School Story was released! The entire studio was excited! It was Pixelberry’s first game, therefore, player impressions were critical.

I was quite nervous about the results, especially with our game performance on untested devices. We kept refreshing our product and customer support pages to see if players were experiencing any issues.

Current number of devices supported on Google Play store

Weeks and months passed by and only .01% of our players reported that the game did not run well on their devices. After following-up on these issues, we found that the lowest-end devices tended to perform poorly, no matter how hard we tried to optimize the game for them.

As this was a very small percentage of our player base, and the overall game experience was a positive one, we concluded that the launch of our first game was a huge success!

Through partnerships with non-profits, Pixelberry's hit game High School Story has taught millions of players about tough teen issues, like cyberbullying and eating disorders.

You can find out more at regularly posts content from a variety of guest writers across the games industry. These encompass a wide range of topics and people from different backgrounds and diversities, sharing their opinion on the hottest trending topics, undiscovered gems and what the future of the business holds.