Virtual reality (VR) and 360º experiences have the power to create stunning visual worlds that surround the viewer. But to create a truly immersive experience, you need to consider the sound design too. Audio can make or break an experience – and often this is overlooked. Jamie Currie, Lead Developer and audio engineer discusses the importance of sound design in 3D spaces.

Why is audio so important?

A lot of VR and 360º experiences have audio that’s headlocked, which means when you turn your head, the sound moves with you. But of course this isn’t what happens in the real world – and the whole point of VR is it’s meant to be as close-to-a-real-world scenario as possible. So straightaway, if your audio is headlocked, the realism of the immersive experience is undermined. Audio should therefore match your environment – which ranges from the right amount of reverb for the setting, to the sound matching the direction of its source. In the VR industry, this is referred to as spatial audio.

Spatial and binaural audio

Spatial audio is the term given to a full-sphere surround-sound technique that uses a dimensional approach to audio, which mimics real world sound. Also known as ambisonics, this type of audio makes the sound in a VR or 360º experience much more realistic. Binaural audio aims to achieve the same effect, but it’s optimised for use with headphones – which brings us to the next consideration.

Headphones or speakers?

For VR or 360º, this depends on the type of experience and the context in which people will use it. For individual experiences or any experience delivered in-engine, headphones work well – with the audio using the telemetry from the headset to mix the volumes of the different audio channels to give that spatial experience. This could apply to a game or marketing experience, and it’s good for interactive experiences too.

However, there are benefits to using a surround sound setup instead – where you effectively create a real world spatial mix that lets your ears do what the software would otherwise do for an ambisonic file. Not using headphones can actually be useful for making people feel at ease when they’re using a VR experience because they’re not shutting off all their senses, while still getting the realistic experience of sound all around them. I recently worked on a project for Mazda where they used a 5.1 surround sound setup for a VR test drive that toured around Europe. By avoiding headphones, they were able to get more people through the experience while still using audio that places the users in the virtual moment.

But it’s different again if you introduce more than one user, for example in multiplayer scenarios. On one hand, a surround sound setup enables the players to talk to each other more easily without having to use microphones, but using a single sound source introduces more room for inaccuracy with regard to where each person is placed in the environment, which you wouldn’t have with headphones. You also have to consider where the experience is being used and what risk there is from sound pollution.

Give yourself time

Whether you choose to use binaural audio through headphones or spatial sound through speakers, the most important thing is giving yourself enough time to create the right mix. Creating the right audio track is a subtle skill that makes a big difference and it should be carefully considered whenever you’re creating a 3D experience. Often it’s overlooked – hence why so many VR experiences have headlocked sound – but if we start adding in enough time to build schedules to allow a thorough and accurate audio sound mix, we can only make more realistic, more immersive and ultimately better VR and 360º experiences.