Presented by Dolby

Think of of spatial audio and video game sound design may first come to mind. Yet, this technology is also valuable for live audio in teleconferencing and virtual meetings. In both cases, spatial sound is a powerful tool for creating immersive experiences.

At GamesBeat’s recent “Into the Metaverse” event, Philip Rosedale, CEO & co-founder at High Fidelity, Andy Vaughan, senior developer relations manager at Dolby, and Mike Minotti, writer at GamesBeat, gathered for the session “Listen to the metaverse” — and discussed some of the challenges of bringing the engaging qualities of spatial sound to everyday events like meetings, online corporate functions, and even virtual conferences.

Traditional audio — even 5.1 and 7.1 surround — is flat. It only provides audible cues within a 2D (front, back, left, and right) environment. Spatial audio, in contrast, gives the illusion that sounds are coming directly from an object in a three-dimensional space.

Let’s consider that for a moment. Not only is sound represented accurately in three dimensions rather than just two, but the spatial nature also enables anchoring sound to a moving subject, such as a character in a game or a presenter at a conference.

Let’s take a look at some of the issues around engaging audio experiences, for both gaming and live events.

Spatial sound for gaming

Players seek a realistic, immersive, multi-sensory gaming experience in their virtual worlds. And, let’s face it, they take as many advantages as they can to beat their opponents. Spatial sound helps on both fronts.

Consider the audio space currently represented in most gameplay. Sound can come from the front or back, depending on speaker location. While games attempt to represent sounds originating from above or below the gamer, or from a character moving through the game space, developers are still limited by fixed-speaker audio.

Spatial sound, whether delivered through headphones or speakers, incorporates sound into the storyline and expands the game space from two to three dimensions. 3D sound creates a broader field of awareness than visuals alone, immersing the gamer into the sound space.

Think about how sound immerses you in the real world versus the traditional surround sound world. In a thunderstorm, with traditional sound, the thunder is around you. In the real world, and in the spatial sound world, thunder emanates from above then surrounds you. The sound of rainfall in a game’s traditional sound space comes from around you. However, in spatial sound, you can hear raindrops hitting the ground below you or roofs above your head. These realistic sound sources help immerse gamers in a more familiar environment, helping developers tell interactive stories more effectively.

Spatial sound helps gamers follow characters as they move through space, climb over objects, or dive into holes. The sound mirrors the character’s actions rather than being fixed to speakers. It’s part of the storytelling process.

By anchoring sound and persistence of objects in the game space, spatial sound enables the gamer to pick up cues from sound movement. An essential part of gameplay is reacting to changes in the environment. Consider a threat coming from the player’s upper front left: spatial sound delivers the threat’s audio cue before the enemy enters the player’s field of vision. Anchoring the sound to the object allows the player to more precisely locate their adversary and react more accurately than in the traditional sound space.

Additionally, spatial sound helps make gaming more accessible to visually-impaired players. The greater range of auditory cues provides the visually-impaired gamer with a more complete game space experience, as the sound experience is similar to the visual experience.

Spatial sound for live audio

Live audio sources present their own unique set of challenges.

You’ve probably been in video conferences where you wished the audio quality was better than an AM radio broadcast. A better spatial audio experience can make otherwise bland meetings not only easier to understand, but positively engaging.

First and foremost is the fact that the sound in video conferences is of poor quality. (Or, as Phillip Rosedale of High Fidelity said, “We need audio that doesn’t suck!”)

Another challenge is ensuring that everyone has the same sound experience, based on a standard source, regardless of hardware.

The mind can tolerate 1/5 a second of latency without noticing it. Typical latency on a video conference is two seconds or more. This leads to people talking over each other, because they cannot properly identify pauses in conversation.

On top of that, most Bluetooth headsets do not capture stereo sound through their microphones. And while noise and echo cancellation reduce distracting outside noises, these features also frequently reduce sound quality.

We’d like to have conversations in an audio conference mimic the live experience. For example, an audience member should be able to hear two people have a conversation off to the side while listening to a third person.

The hardware we have today is good enough. The question is: what can the software solutions do with that sound to make a more immersive and realistic experience?

Spatial sound creates the illusion that sound is coming from a three-dimensional space. Rendering participant voices in space with audio helps the brain distinguish between participants and adds to the perception of presence, or the sensation of being in the room with these participants instead of hearing simple disembodied voices.

Sound APIs as web services can offer some solutions for teleconferencing, particularly if we take some lessons from the amazing spatial sound experiences already available in gaming.

Experience spatial sound

Spatial sound enhances a virtual space’s ability to mirror sounds in the real world. While we discussed the advantages of spatial sound for both gaming and live audio events, you should experience it for yourself to fully comprehend its ability to transform auditory experiences.

Dig deeper: Learn more about spatial sound and discover how to create your own immersive audio experience with these resources:

Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. Content produced by our editorial team is never influenced by advertisers or sponsors in any way. For more information, contact sales@venturebeat.com.