Feature

Things I learned about touch control when developing for Kinect

Jennifer Schneidereit on a window to another world

Things I learned about touch control when developing for Kinect
Born and raised in Germany, Jennifer fell in love with video games at an early age when playing Pac-Man at a friend's house.

After graduating from university with a degree in computer science, she started her career at Acquire, an independent Japanese game developer in Tokyo. She initially began as a software engineer, but gradually moved towards game design.

In 2009, Jennifer relocated to the UK, to work at Rare on the BAFTA-award-winning Kinect launch title
Kinect Sports.

In late 2010, I co-founded a small independent game developer called Nyamyam, with Phil Tossell who I meet while working at Rare.

Since then I have been working on our first game Tengami, an atmospheric adventure game set inside a Japanese pop-up book.

Tengami was conceived with the iPad as its primary platform in mind.

But before Nyamyam, I mostly worked on traditional controller-based games with the exception of Kinect Sports, which is purely gesture-based.

Standing in front of their TVs, players run on the spot to make their on-screen character sprint or swing imaginary table tennis bats to smash the ball back at their opponent.

At first glance it might seem like Kinect-controlled and touch-controlled game are very different creatures: one uses full body motion, the other uses only the hands (or in the most extreme case only a single finger).

But it's actually surprising how many things carry across between the two.

So I thought it would be informative to talk a little bit about things that I learnt working with Kinect that have proven useful when developing touchscreen games.

You are the controller

During the development of Kinect Sports, there was a marketing phrase that was doing the rounds: "You are the controller".

At the time it seemed really lame to me, but the intention behind it was surprisingly valuable and something we leaned on heavily with Kinect and subsequently on iPad.



With skeletal tracking being one of Kinect's most hyped features, the phrase made sense. "You are the controller" was intended to convey the idea that there is a direct correspondence between the actions you perform with your body and what you see happening in the game.

This contrasts with a controller where there is an abstract mapping between what you do and the results.

For example to play table tennis with Kinect, the player swings their arm as if they are holding a bat. In a controller-based version of table tennis, the analogue sticks might be used to give direction and a button push would result in the actual performance of the swing.

Direct control

Around the same time that we were finishing Kinect Sports, I heard a talk by Graeme Devine at GDC 2011 in which he talked about making the best use of the new iPad for gaming.

In the talk Graeme touched on a lot of interesting points but one thing in particular matched with what we'd been finding with Kinect.

He said (paraphrasing): "The best touch games are those where you feel like you are reaching into the game world and directly manipulating it".

When I started working on Tengami, this idea of being the controller and reaching directly into the game world became the single most important design idea that we used.



The pop-up book concept is directly derived from this. The player's finger directly manipulates the digital pop-up book world in a very similar way as to how they would use a physical pop-book with their hands.

You are the controller, but a different controller to me

One challenging side effect of "being the controller" is that we are all different.

With Kinect there were obvious differences such as height and body shape, but there were also subtler differences related to age, mobility and experience in performing certain motions.

Children, in particular, struggle with doing any kind of precise movement. All of this made it very difficult to design something that was easy for anyone to play.

What we learnt was that the gestures we used had to be generous but also sufficiently distinct to account for these differences.

In many ways these two requirements are competing because the more generous you make the detection the harder it is to distinguish between gestures.

What it really means is that you should use as few actions as possible in a given context and think very carefully before adding new ones that might conflict.

In Tengami, for the most part there are only two actions: double tap to move the character and slide to open and close pop-ups and the various other mechanisms.



Tengami offers striking graphics as well as strong controls

A double tap is used in preference to a single tap to increase the separation with the sliding action. The double tap is also a much more affirmative action.

What you find on touch, as we found with Kinect, is that people will explore the gesture space. With touch they do this by feeling around the device with their fingers. If a single tap does an important action then this can cause a lot of unwanted triggering of actions.

User test everything

The previous sections are all well and good but the most important thing you need to do is to confirm that your theories are correct.

During Kinect Sports we had access to what is probably the most comprehensive user testing facility in the world at Microsoft.

This was a truly eye opening experience. Each week we would get a fresh set of players in to test the latest build on. We could directly observe them on a live feed or receive videos afterwards.

One of the most humorous moments in development came from user testing. We had built a prototype for basketball that involved bringing your hands together at about waist height in order to catch the ball after dribbling and before shooting.

It tested fine on almost everyone. That was until we came across one particular teenage girl.

By accident she had managed to trigger the gesture by placing her hands on her hips. She exclaimed loudly to her mum that she had figured out that placing her hands on her hips was the correct way to catch the ball!

Watching 30 minutes of this was the starkest lesson I ever had of the importance of user testing.

Obviously now as a small indie studio, we don't have access to this kind of facility any more, but we use any opportunity we can to try the game on new people.

Game shows such as PAX are a great way of getting a large number of people to play the game. It is worth attending a show like PAX for this alone.

Summary

In closing, try to think about the screen on your touch device as a window into another world. It's as if you're Alice reaching through the looking glass into a world that exists on the other side.

Thinking about it in this was will lead to a lot more joy and wonder for your players.

On top of that keep your interactions few in number and separated as much as possible.

Lastly test your game as often as you can. Ideally you should use people you don't know, but if all you have are friends and family then by all means use them. And if you can afford to go to shows, then I'd highly recommend it.

You can keep up-to-date with Nyamyam via its website and Twitter.

PocketGamer.biz regularly posts content from a variety of guest writers across the games industry. These encompass a wide range of topics and people from different backgrounds and diversities, sharing their opinion on the hottest trending topics, undiscovered gems and what the future of the business holds.