By Winifred Phillips | Contact | Follow
Hello there! I’m video game music composer Winifred Phillips. Lately, I’ve been very busy in my production studio composing music for a lot of awesome virtual reality games, including the upcoming Scraper: First Strike first person VR shooter (pictured above) that’s coming out next Wednesday (November 21st) for the Oculus Rift, HTC Vive and Windows Mixed Reality Devices, and will be released on December 18th for the Playstation VR. My work on this project has definitely stoked my interest in everything VR! Since the game will be released very soon, here’s a trailer video released by the developers Labrodex Studios, featuring some of the music I composed for the game:
Scraper: First Strike is just one of a whole slew of VR games I’ve been working on over the past year. Last year, when I was just starting to get really busy working with VR development teams, I wrote an article here that offered a bunch of informative resources connected to the field of VR audio. The article I posted in 2017 took a general approach to the role that audio plays in Virtual Reality experiences. Since we’re well into 2018, I thought we could benefit from expanding that topic to include the state-of-the-art in VR headset platforms. Taking a look at the hardware platforms that are currently available should give us video game composers a better idea of the direction that VR audio is currently headed.
For one thing, VR is now broadly considered a part of a larger category that also includes AR (Augmented Reality) and MR (Mixed Reality) devices. Those two categories are often considered synonymous, although that’s certainly debatable. Since there’s no clear expert consensus at this point on what characteristics separate AR from MR, let’s just consider them as one category that we’ll call AR/MR for now. In this article I’ll be focusing on resources that are specific to each of the competing platforms in VR and AR/MR.
Let’s get started!
Audio for VR and AR/MR devices
A wide variety of head-mounted devices now exist that can immerse us in imaginary worlds, or bring fantastic creatures to life in our living rooms. While many of these devices share common underlying technologies in regards to audio creation and implementation, there are differing tools and techniques that apply to each of them. I’ve included links in the discussion below that may be helpful in understanding how these technologies differ.
When virtual acoustics meets actual acoustics
The newly-released Magic Leap One is an AR/MR device. This means that it allows the wearer to see the real world, while superimposing digital images that seem to exist in reality, and not just within the device. For instance, an AR/MR device can make us think that a miniature toy dinosaur is toddling across our coffee table. With this in mind, creating audio for AR/MR becomes a little tricky.
It’s a fascinating problem, and one that the Magic Leap folks have considered seriously, using a system they’ve dubbed ‘Soundfield Audio’ to apply physics calculations that can produce appropriate acoustics based on the environment. They’ve also patented a spatial audio technology that uses the wearer’s head movements to calculate the position of virtual sound sources. Here’s a video that shows off a video game music visualization application for Magic Leap called Tónandi:
The Hololens is also an AR/MR device, and therefore faces a lot of the same issues as the Magic Leap One. To address these, Hololens uses a spatial audio engine that calculates the position of sound-emitting sources combined with personalized Head Related Transfer Functions or HRTFs (a concept we discussed in an article from 2015). These HRTFs help to localize all the aural components of the virtual soundscape. In addition, the Hololens creates a room model to match the user’s location so that sounds seem to reflect from real-life walls and travel convincingly to the player’s ears. We should expect this technology to improve when Microsoft releases their next generation of Hololens early next year. Here’s a video produced by Engadget that goes into more detail about the audio experience delivered by Hololens:
Spatial sound for mixed reality
Positional audio inside the virtual machine
Now let’s move on to discuss what’s happening with the current VR devices. As we know, unlike an AR/MR headset, a VR device cuts us off completely from the outside world and plunges us into an environment existing entirely within the machine. There is currently a healthy and varied crop of VR devices from which to choose. The two most popular and famous VR headsets are the Oculus Rift and the HTC Vive. 
Standalone VR audio
Now let’s take a quick look at the standalone VR devices (i.e. those devices that don’t need to be hooked up to a computer or console, and don’t need a mobile phone installed in order to work). These VR headsets offer untethered, cable-free virtual reality exploration, but they’re also usually a bit less powerful and full-featured. The five best-known standalone headsets are the Oculus Go, the Oculus Quest, the Lenovo Mirage Solo, the HTC Vive Focus, and the Shadow VR.
The Oculus Go and Lenovo Mirage Solo both hit retail this May. The HTC Vive Focus and the Shadow VR both became available for consumers just this month. The Oculus Quest was recently announced and is expected to hit retail in spring 2019. All five use a Qualcomm smartphone processor chip from the Snapdragon line, so in that respect they’ve essentially adopted the internal mechanism of a high-end mobile phone and simply incorporated it into their on-board hardware. In fact, the Qualcomm Snapdragon 835 
Audio for mobile VR
If you do a quick Amazon search under the phrase “Mobile VR Headsets,” you’ll see that there is now a dizzying plethora of headset models based around the “insert your mobile phone here” philosophy. These headsets all rely on the processing technology of the phone inserted into them, and there are so many varying models that I won’t be attempting to delve into that topic. Generally speaking, if there is a SDK specific to a particular VR headset model, then it should be considered. For instance, the Oculus Audio SDK makes sense for the Samsung Gear VR and the Oculus Go and Oculus Quest, since all three are Oculus-designed VR systems. Likewise, the new Resonance Audio SDK from Google is a good choice for any of the Google headsets (Daydream View, Google Cardboard, Lenovo Mirage Solo with Daydream). Here’s a brief video produced by Google that demonstrates the Resonance Audio SDK:
Conclusion
That’s our discussion of where things currently stand with regards to VR platforms! In the next article, we’ll be focusing on tips and tools for game audio folks working in VR. I hope you enjoyed the article, and please let me know your thoughts in the comments section below!
