How Sound Works & How We Perceive It
Overview
Games are usually an emulation of the real world so understanding the principles of audio will help you in all aspects of the sound design and even the implementation process.
Whenever I start designing a new sound I ask myself “How is (or would) this sound be created in the real world?”
In this mental exercise, I try and pick the sound apart into its various mechanics and elements to get an angle from which to approach the problem.
Finding these angles is often the hardest part so let’s try this exercise:
Imagine you are given the task of creating the movement sounds and footsteps for a robotic dinosaur creature from the game Horizon Zero Dawn. Think about how the sound is made:
What kind of materials are moving together when the creature moves?
How big is it?
How heavy is it?
What surfaces can it walk on?
All these questions will help us figure out the sounds we want in our palette.
But we can go even further, and think about the actual physical properties of the sounds - the individual changes in air pressure, and how they happen in the first place.
Let’s look at another example, this time of a gunshot. We have:
Mechanical parts that bring the bullet into the chamber and trigger the explosion
The muzzle blast that propels the bullet forward as the firey hot air pushes out the barrel
The crack caused by a sonic boom as the projectile leaves the barrel, and the whizzing sound as it travels toward the target
The reverb of the waves bouncing around the environment, which provides us with the context of the environment the sound is in
And lastly, the foley, which means the sounds of the gun being handled and manipulated by the user
All of these layers combined make up a great gunshot sound, but we can change and adjust each one individually to alter the feel of the weapon.
By swapping out a clean mechanical “chk chk chk” for a very rattly “clak clak clak” we can turn a modern high-precision weapon into an old, rusty piece of nuts and bolts barely holding together.
Changing the muzzle blast from a loud “whack” to a low “thump” will give the impression of a gun with a suppressor, and modifying the reverb will change the space we perceive the sound to be in.
As you can see, understanding the fundamentals of how sound behaves in real life can make it easier to create awesome sounds by analysing the individual components that make up a single effect.
So without further ado let’s get into the first topic.
Acoustics - How Sound Is Made
Acoustics is the field of study that examines how sound works and behaves in different environments, as well as how it moves through various mediums.
First let's look at this old but fantastic document for a basic overview of the topic:
[LINK TO OVERVIEW]
Now we'll dive a little bit deeper with this crash course on the physics of sound:
https://youtu.be/qV4lR9EWGlY
Lastly a little overview of acoustics in rooms:
https://youtu.be/JPYt10zrclQ
Game Audio Application
As you can see in the videos above, there are a lot of acoustical phenomena that we often knowingly or unknowingly try to recreate, such as the Doppler effect.
Another very obvious one is reverb, which gives the player important information about the space that they are in. It’s important that the reverb matches the acoustical properties of the space so that the expectations of the player are met and we don’t break the immersion.
The simplest way to do this is to manually add reverb to the sounds inside of your DAW (Digital Audio Workstation). However, there is a big drawback to this as you can't control the amount of reverb after the sound is exported and implemented into the game.
Thankfully most game engines and audio middleware allow us to add reverb to sounds in real-time while the player is playing and adjust it depending on the space they are in.
There are some excellent but very advanced articles/videos on how this is done in various games which can be found below, but I recommend moving on to the rest of the learning roadmap for now and revisiting them later.
Further Reading (Advanced Acoustics)
These articles and videos below are more advanced and dive deep into the technical systems of games. Feel free to read/watch if you're interested, but it's okay to come back to this later once you have a better understanding of how game engines and audio middleware work in the first place.
Properties of a Sound Wave: https://youtu.be/KUMI9sqD6vc
Phase cancellation: https://youtu.be/NaK6GrLg4cE
Decibels (dB): https://youtu.be/F4r3WI-JXlc
Psychoacoustics - How We Perceive Sound
Sound can influence emotions and trigger involuntary evolutionary behavioural responses (manipulating the player through sound to think and feel certain things). This is an incredibly powerful tool for sound designers which we can harness by learning the theory behind psychoacoustics.
But first, what is psychoacoustics in the first place? This article and TED Talk are a fantastic starting point: https://www.larecordingschool.com/psychoacoustics-the-psychology-of-sound/
Game Audio Application
Now that we have a fundamental understanding we can look at some examples of games that have harnessed this.
How Frostpunk utilizes psychological tricks in its UI sound design:
https://youtu.be/kbjYl8m5cOI?t=240
A number of great articles by Denis Zlobin on psychoacoustic phenomena:
https://deniszlobin.medium.com/
Further Reading (Advanced Psychoacoustics)
Similar to Acoustics, these are some more advanced resources to check out when you’re ready!
Casey Connor’s “Audio Illusions” series on YouTube:
https://youtu.be/OiW8gzBGz1A
Vox’s video about Shepard Tones, the audio illusion used to create the “endless riser” effect:
https://youtu.be/LVWTQcZbLgY
