VR for the Game Music Composer – What’s New?

Recently I bought my first VR headset, and since then I have been adventuring in the world of virtual reality courtesy of Google Cardboard. For something as high tech and impressive as the VR experience, Google Cardboard makes the whole process easy, low-cost and accessible.  While Google provides instructions for users who’d like to make their own headsets from scratch using simple craft materials, I opted to purchase a version made by the good folks at IMCardboard.com.  Pictured to the right, you see me wearing their EVA 2.0 headset, made from a rubber-like material that’s very comfortable and lightweight.  Despite the more sophisticated look and materials, this headset still adheres to the Google Cardboard specs in terms of its design.  Coupling the immersive visuals offered by this headset with powerful music and sound from my trusty pair of Sennheiser HD 650 headphones, I was now ready to go adventuring in cyberspace.

I don’t know if 2016 is going to be the year of virtual reality, but since I’ve taken my first step into the VR world, I thought we could use this blog to touch base with developments in the VR world.  We’ll look at a brand new audio tech conference that should be particularly interesting to VR folks.  We’ll also get an overview of a couple of top audio technologies for virtual reality video games.  One of these new technologies pertains directly to Google Cardboard, so that’s where we’ll begin:

It wouldn’t be make believe…

First, let’s get a little historical overview.  A good place to start may be with a popular old song that seems particularly appropriate right now:

“It’s only a paper moon… sailing over a cardboard sea… but it wouldn’t be make believe if you believed in me.”

This music rings true at the moment, because Google has taught us all to believe in cardboard.

In June 2014, Google announced their new virtual reality initiative called Google Cardboard, and a lot of people didn’t believe in it (they thought it might be a joke).  Over a year and a half later, nobody’s laughing.  Google just created a dedicated VR division with a new VP to run it, and 5 million Cardboard headsets are currently out in the marketplace. While some say that the VR experience delivered by this device is limited at best, it has opened up the awesome visual spectacle of virtual reality to the masses. (Personally, I’ve been enjoying my initial forays into VR with Google Cardboard, and I’m looking forward to broadening my virtual horizons).  While 3D visuals have wowed users of Google Cardboard, the enveloping audio of a true VR experience has been unavailable to users of these headsets – until now.

On Jan. 13, 2016, Google announced the introduction of spatial audio capabilities for the software development kits designed for Google Cardboard Unity (iOS) and Google Cardboard Android.  The underlying technology was developed by ThriveAudio; a group of research engineers from Trinity College Dublin who specialize in 3D virtual audio environments. While previously focusing on the audio needs of the Oculus Rift VR device, ThriveAudio was acquired by Google in July 2015 and set to work bringing spatial audio to Google Cardboard.

In an interview with journalist Drew Olanoff of TechCrunch, Google Cardboard’s project manager Nathan Martz stated that the spatial audio technology would “allow creators to control the direction of sounds in an immersive experience.” Martz explained, “Sounds that come from the right will reach a user’s left ear with a slight delay, and with fewer high frequency elements (which are normally dampened by the skull).”

It’s good news that the most famous and accessible VR platform is getting spatial audio – it shows how serious Google is about entertaining its Cardboard users.

Introducing AMBEO

At the recent Consumer Electronics Show in Las Vegas, the German audio company Sennheiser (my favorite headphones manufacturer) unveiled a suite of immersive audio technologies, dubbed AMBEO.  The proposed AMBEO product line is designed to provide solutions for three dimensional audio in multiple listening situations.  While AMBEO has applications in the world of complex surround speaker systems, the emphasis of this new technology line is clearly on headphones.  In fact, AMBEO represents Sennheiser’s ambitious entry into the world of spatial audio for VR.

Much of the AMBEO tech on display at CES was in an early prototype phase, which intrigued journalist Tyll Hertsens, editor in Chief at the audio technology site InnerFidelity.  He draws a parallel between Sennheiser’s bullish attitude towards VR sound and the threat of imminent competition from Apple.  “With the acquisition of Beats,” Hertsens speculates, “Apple now has all the pieces in-house to develop the technological ecosystem to commercialize immersive audio… (Sennheiser needs) to get things moving along. And that’s why they produced a technological demo at CES, to wake people up and get their creative and competitive juices flowing.”

The AMBEO VR audio technology for headphones was presented in a three-pronged approach.  The first was the Venue Modeling software that could faithfully replicate room acoustics, allowing DJs to play their music sets from within several dance clubs around the world.  The second was a virtual reality microphone designed to capture audio in four quadrants (launching Q3 2016).  Finally, the third was a proprietary Sennheiser algorithm designed to enable video game developers to create three dimensional soundscapes for VR.  At the Sennheiser booth at CES, this algorithm was put to the test in the virtual-reality game EDEN, developed by SoulPix.  The video game is still in development, but an early proof-of-concept demo is now available for download for the Oculus Rift. Here’s the trailer for EDEN:

2016 AES International Conference on Headphone Technology

Finally, a brand new conference organized by the Audio Engineering Society should provide lots of useful takeaway for game audio pros working in VR.  The International Conference on Headphone Technology will focus on “technologies for headphones with a special emphasis on the emerging fields of Mobile Spatial Audio, Personal Assistive Listening, and Augmented Reality.”  Of particular interest are the proposed mobile spatial audio discussions, which should directly pertain to VR projects for mobile platforms such as Samsung Gear VR and Google Cardboard.

Well, that’s my blog this week!  If you’ve encountered any interesting developments in VR audio tech, or if you’re taking your first trips into the world of VR, please let me know in the comments!

---

Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.