Spatial audio in 360 degree videos

There is more and more demand for good VR videos. With the first videos we still had to deal with fuzzy images, but you can’t get away with that anymore. And of course, VR is more than just images. The audio, just as much as the footage, needs to be as realistic as possible. Google says: “Realistic sound is essential to make the VR experience believable.” A good enough reason to take a closer look at this. What do you need to be thinking about when it comes to 360° degree audio?

What is 360 degree audio

What exactly is that? With 360° degree audio you hear the sound all around you. Not just left and right, but also in front of you, under you, behind you and above you. Is there someone standing behind you who says something? Then the voice needs to come from behind. And, if you turn around..? Then the sound needs to turn with you, so that the voice that just came from behind you, now comes from in front. Just as it does in real life. An aeroplane that flies over, you really need to hear above you. First distantly and then, the closer it gets, the more clearly you hear it. That’s called Spatial Audio.

Ear candy

With VR it’s important to bring in the audio as early on in the process as possible. As a video maker you will immediately be thinking about beautiful shots, you can also start thinking about beautiful soundscapes straight away. A dripping tap in an empty house, a smouldering fire for a walk through the woods. Or if you’re making a recording of a live performance by a band, then it might be nice, as a virtual visitor, to first experience coming into the venue. Outside on the sidewalk you can already hear the deep bass notes rumbling through the walls.

And you will also want to think about ambient sound. A city without the hustle and bustle is no city. A beach without the sound of the surf isn’t a beach but a lake. A wood without birds or the rustle of leaves is no fairytale forest, but a spooky one. In other words: the sound determines whether you really feel you are somewhere else, or not.

Close your eyes…

An easy way to get ideas for sound is to imagine you are making a radio play. Let your imagination… speak. Close your eyes. We are on a sunny beach. What do you see? And now, what do you hear? Right in front of you the waves are breaking on the beach, and there are seagulls circling over your head. In the distance you can hear a group of children playing. A speedboat races by, and behind you the lifeguards’ flag is flapping in the breeze. In your imagination you naturally hear some things very clearly close by, and others further away. If, for instance, someone is standing behind you and begins to speak to you, you would immediately turn around. It’s much nicer if you already hear the person start talking from behind you.

How does 360 degree audio work?

The spatial audio mix that is made for VR or 360° videos, uses a technique that purposefully deceives the ear. Naturally, the sound of an aeroplane flying overhead doesn’t really come from above, but just from the earpieces of your headphones. With normal speakers you can’t usually hear this effect, or not nearly as well. This is because, among other things, the ear/brain listens for the sound bouncing off objects (like walls) to determine where the sound is coming from. If you play the sound via speakers, the sound rebounds of the walls of the room you’re sitting in too. That means you lose part of the audio mix, and so the 360°, illusion.

How do you record 360 degree audio?

There are special microphones to record 360° degree audio, but you can also make use of old school mics and sort the 360° degree experience in post.

In production you need to take account of audio from the get-go. In the period that camera shots were no broader than a TV screen, you were able to easily hide all necessary recording equipment and the sound guys behind the camera. But yeah, if your audience has a 360° degree view then all they have to do is turn around to see your colleagues and all the gear…

So, if you use a special 360° degree audio mic, then you will need to hide it somewhere. And, to make sure that the audio makes sense, the mic will need to be next to the camera. Only thing is, there’s a good chance that not all the sound will register well, or that you pick up noise from the cameraman.

In addition you will want to emphasise certain sounds. Similar to the way that you zoom in with a camera, you will want to zoom in on particular noises. In these instances you are better off using close mic’ing. You can of course record a speaker with a wireless mic. It gets more complicated if you want to emphasise the flapping of the flagpole, and can’t see any possibility of where to hide the mic. In those cases you can record the sound separately from the image with a close mic, or replace it later with something from your sound effects library.

How do you mix it?

In principle, you are able to make every audio recording ‘spatial’ in the edit. To do that you will need software that can ‘pan’ the sound not only from right to left but also from above, under, behind or in front of you. There are different tools on the market for this, we work with 360° Ambisonics Tools from Waves. They have also developed a cool ‘head tracker’ which registers the user’s head movements and adjusts the audio to match them. That mixes a lot more easily. There are also free tools, like that developed by Facebook, 360° Spatial Workstation. Google has a great site with more info about Spatial Audio.

And what do we do with the voice over?

A voice over often has to just ‘stand still’, because he/she is not in shot. It can feel strange if the sound of a voice over is really easy to pinpoint, because at the point where the sound originates there’s nothing to be seen. For the same reason you often want to hear the background music in stereo alone. But if the band playing the music is in view, then you do want a spatial mix. There are lots of videos that let you see and hear concerts like this.

So… make sure that the sound for Virtual Reality – just as the footage – is as realistic as possible. Remember it’s important that you take account of sound during the development phase. Some of the things that you can’t capture when recording the footage you can add in later. Make sure that when editing you work with software that allows you to make the audio spatial. Do you want to know how you can make use of Spatial Audio in your production? Or do you have other questions about online Audio Post Production? Then our Chief of Sound, Gijs Friesen, will be happy to help. Mail him on: gijs (at) voicebooking.com, or leave a comment below.

Which factors are influential for your in-house production success? We have 15 practical tips – The do’s and dont’s of in-house production. 

Written by Gijs Friesen & Jente Kater