Comment & Opinion

Mobile tones: The audio tool and service providers pushing the sound of smartphone gaming

We speak to the top audio companies about creating engaging sounds and music for the small screen

Mobile tones: The audio tool and service providers pushing the sound of smartphone gaming

Over in the world of console and PC gaming, playing while using headphones has become something of a mark of respect for the value and importance of quality game audio.

That credibility is the reason vastly expensive gaming headphones exist, even though most players don't own a pair. Indeed, such gaming peripherals are something of a status symbol to demonstrate how seriously a given user takes playing on a tethered platform.

On a related note, increasing numbers of games today boot up with the menu prompt 'please play with headphones for the best experience'. And you don't need to spend long in VR to realise how essential quality audio is to that emerging form.

And yet in the realm of smartphones, where headphones are much more commonly used by the lay-user, game audio is too rarely talked about. There's been plenty of astonishing mobile game audio, of course, especially in near exclusively sonic works like Somethin' Else's Papa Sangre series.

Perhaps it's tied to the wider credibility issue mobile gaming suffers generally, or maybe it's a demonstration that the best creative specialties - as with special effects in cinema - have triumphed when they are convincing enough not to stand out. Either way, mobile audio rarely enjoys the credit it deserves.

Not that that has stopped the gaming audio technology providers from continuing to innovate in an effort to better serve mobile customers. And that is because they know more than any just how important sound is to a mobile game's success.

Rings true

"Good audio is necessary for any commercial project, be it video games, films, or music," asserts Simon Ashby, co-founder and VP of technology for Audiokinetic, the giant of games audio middleware behind the prolifically adopted Wwise solution.

"I don't see any differences for mobile games specifically. You can back this argument by observing that most successful games have great audio, but successful games with notoriously bad audio are rather rare. If you want your game to be successful, you need to invest in audio, both financially and by backing up the audio artists with the right tools and programming support."

If you want your game to be successful, you need to invest in audio.
Simon Ashby

Put so simply, it sounds – well – rather simple. By their broadest definition, video games are interactive worlds built from light and sound. So if sound is half of the very form, it deserves something at least close to 50 per cent of the effort.

That divide between light and sound warrants a little closer inspection, though. While smartphones are often celebrated as 'supercomputers in the pocket', they do have relative limitations; especially where visual rendering, image fidelity, 3D assets and animation nuance are concerned.

Sure, audio can be half the offering of a mobile game, but it might be the less limited half, meaning sound can be used to not only buoy up visuals, but also to balance a higher production quality tone overall. Of course, there is another element of mobile games that isn't just seen or heard. Here, gameplay is ultimately king.

But even in the case of hypothetically perfect gameplay, sound can provide an edge valuable enough to make another standard-bearer of quality game audio see the mobile space as worthy of supporting.

"With limited display size and low quality speakers, mobile games have tended to focus on game play to engage users," says Matthew Connor, general manager at Firelight Technologies, which is responsible for the hugely popular FMOD sound tools.

"As devices improve and more players have headphones, developers are increasingly using audio to enhance the experience. In an extremely competitive marketplace, games with great sound effects and engaging music are going to standout from the crowd."

MZ has used FMOD for games such as its hit title Mobile Strike

Quality audio also lets mobile games do something typically associated with PC and console; delivering the cinematic and narrative-lead experiences. At least, that's what the team at Tazman-Audio believe. They are the longstanding creators of audio solution Fabric, which currently enjoys much popularity as a mobile game audio toolset for Unity.

"Game audio provides depth and narrative to gameplay, which directly immerses a player to engage and own their role," offers Tazman-Audio's director of sales and business development Suzie De Boeck.

"Audio is a way in which the design of the game reaches all levels of players, from the casual to the professional player. As with a movie, the audio narrative of a game can provide the psychologically important mood setting, the suspense, the humour and fundamentally provides the tempo and rhythm of intended gameplay.

"For mobile games, and increasingly, the integration with VR games, audio is an essential factor in creating the experience, thus raising the bar for game quality as well as playability and overall perception of the game.

"Simply put, well planned audio can direct the player to respond and feel the game in a way the design demands, thereby increasing the engagement and ultimately, significantly but discreetly affecting the potential success of a game.

Sounds encouraging

The reasons the major audio tool and middleware outfits have accelerated their move on mobile seem obvious. There is much to offer. And that trend is mirrored in the related discipline of audio services, where the likes of Somatone Interactive provide sound designers, composers, voice over producers and implementation experts to game companies of every size.

My motto is 'no more loops', and this means that we should be scoring games with sound and music that adapt to the player experience.
Dale Crowley

"Imagine watching Star Wars without the music," ponders Somatone Creative Director, Dale Crowley. "That would be a very poor user experience.

"Now imagine watching Star Wars with a one minute royalty free loop you found somewhere on the internet. Imagine if every sound – every laser gun – used exactly the same source file. I know it sounds absurd, but that is what a lot of mobile games do. My motto is 'no more loops', and this means that we should be scoring games with sound and music that adapt to the player experience."

Fortunately, for Crowley and mobile games developers generally, that shift in approach is presently underway, lead by the tools that make it possible.

"Audio tools have made huge strides recently," he enthuses. "At Somatone, we have seen a big upswing in the adoption of these tools in the top mobile games we are working on. In the past, audio designers and composers used to just send audio assets over the fence to the developers."

Now – Crowley is relieved to reveal – his peers and colleagues are much more deeply engaged and highly involved in the audio design and implementation process.

"This allows us to do so much more than before, and speeds up the workflow when collaborating with producers and programmers," he continues.

"We talk with programmers daily, and even help with the coding of these sounds directly in Unity and Unreal. Middleware sits right in between the code and the sounds, and allows us to manipulate those sounds in creative and unique ways. It is thrilling to be able to do things with sound that were not possible in the past, and to be able to deliver music that adapts to the player’s actions."

And the reason audio tool providers have stepped up to provide this platform? It will surprise almost nobody that the commercial health of mobile has made such a move irresistible for business-savvy middleware outfits.

"Mobile gaming is there to stay, so everyone has to ramp up their game and make sure they provide the best experience for mobile platforms," says Audiokinetic's Ashby.

"That's true for content providers and tool providers. At Audiokinetic, we make sure that Wwise offers the same features for mobile as for any other platform. The key to our success is that Wwise provides all the flexibility to gracefully scale up or down the audio content and performance usage to adjust to the lower and the higher platform specifications."

Playdead's PC, console and iOS game Inside uses Audiokinetic's audio tools

Tazman-Audio's De Boeck adds: "It’s an exciting time for game audio right now. Certainly we at Tazman-Audio are really pleased with the progression of the industry.

“We see collaborations and sharing of services becoming a driving force within the market and this will open up the opportunities and need between the service providers and the toolset providers, as well as production teams to incorporate the newer mindset of the ever-changing dynamics of game development in general."

Sound studios

Of course, all the software studios, audio engines, composers and musicians in the world can embrace mobile game audio, but if developers and studio heads themselves don't do the same, that movement may be rather meaningless.

A persistent notion has existed longer than modern mobile gaming states that audio is often tagged on in both a game's design process and technology process; an after thought not given the time, money or resource needed to make it sing.

We see collaborations and sharing of services becoming a driving force within the market.
Suzie De Boeck

"Good audio design is holistic," opines Crowley, considering how mobile developers themselves are giving audio the credence it deserves.

"Audio is critical to the story and emotion of any game, and can actually inform game design features and decisions. It is critical to start thinking about and planning for audio as early as possible in the project, ideally from day one.

"It is much harder to shoehorn audio middleware into the code near the end of the project than it is to plan for it from the beginning. Working with audio middleware also allows for sound designers and composers to start work from the start, without needing a build of the game to test out how the sounds will interact with one another and the music.

“Sounds often have to interact with animation systems, 3D environments, and so on. Having a framework in place keeps the workflow smooth, and allows for data abstraction - meaning that the audio can develop in parallel with the code and features of the game."

There's plenty to get right then, and the consensus is that today it happens more than ever before. Still, a long-established rule of thumb for getting game audio right appears as important to mobile as any other form, and Crowley isn't alone in encouraging it.

"Involve the audio content providers at the early stage of the development so they can help reinforce the game design objectives," Audiokinetic's Ashby implores.

"Also, make sure they provide both the audio files and the sound integration: It's the best way to ensure each audio and music asset they produce will be played as intended, in the game."

Tazman-Audio's Fabric was used for audio in The Walking Dead: No Man's Land

These are golden rules of game audio, of course, but just because they are well established in console and PC, doesn't mean they aren't forgotten, especially when so much of the focus in mobile design can fall around monetisation and retention strategies.

Small speakers, huge potential

Over at FMOD HQ, the team is equally keen to remind developers to integrate audio early in the development cycle of a mobile game. But they equally have some advice for teams pondering what else they can do to make their smartphone creations harness the potential of quality audio.

"Consider how audio can be used to immerse the player in a world that extends beyond the screen by creating a sense of the environment," Connor advises.

"Sound can also reinforce or even supplement the visual cues where screen space is limited. Also, ensure the audio is not too harsh or repetitive, especially over long periods. Subtle variations in sounds effects and well-designed adaptive music can make a big difference. This can all be done without coding in FMOD."

Avoiding coding would have once turned up the noses of devoted game makers, but in mobile and the wider gaming ecosystem, working without coding skills is increasingly common. And when there's not the budget for a devoted audio team, that kind of strength can make an audio tool all the more relevant to mobile game development.

Consider how audio can be used to immerse the player in a world that extends beyond the screen by creating a sense of the environment.
Matthew Connor

Connor ultimately concludes by offering one extra tip for studios: "Try mixing on a range of devices and headphones; not just high end studio monitors. With FMOD its possible to connect to the game and mix the audio while it’s being played on any device. This is truly the workflow for game development and can save a lot of time."

While the mobile hardware ecosystem may be far more standardised today than in the J2ME days, it remains a pertinent point.

In mobile, it's extremely unlikely your users will be enjoying your audio efforts via high end multi-speaker audio systems, so be sure to design with tiny in-ear headphones as well as more mid-range options; and smartphones' onboard speakers.

The final word of advice, meanwhile, goes to Tazman-Audio's De Boeck, who strikes a pragmatic endnote on which to consider the importance of audio in mobile and the opportunity presented by middleware and service providers' increasing move to support the field.

"Game audio is a hugely underestimated factor of influence of a player's engagement with a game," he states.

"Players use games to escape for minutes to hours at a time. By giving them the audio experience that not only compliments but lifts the game you can prolong the interaction time, the longevity of the game and the overall potential success of your game. Choosing the cheaper, quicker, familiar solution to game audio should not come before the quality of the product you intend to provide.

“By neglecting the audio you will miss out on the underestimated effect, and you will lower your potential for the game. Trust the game audio to a solution provider that really knows the industry, that believes in game audio and above all, is passionate about it.

"It does need to be thought about at the budgeting stage, with appropriate thought to solution options and requirements."

Will Freeman is the former editor of trade publication Develop, having also written for the likes of The Guardian and The Observer.