In part one we discussed how motion tracking lets us be awesome air guitarists and drummers inside the virtual space. In this article, we’ll be taking a look at how the same technology will allow us to make interesting music using more serious tools that are incorporated directly inside the VR environment – musical instruments that exist entirely within the VR ‘machine.’
Our discussion to follow will concentrate on three software applications: Soundscape, Carillon, and Lyra. Later, in the third article of this ongoing series, we’ll take a look at applications that allow our VR user interfaces to harness the power of MIDI to control some of the top music devices and software that we use in our external production studios. But first, let’s look at the ways that VR apps can function as fully-featured musical instruments, all on their own!
Soundscape
Let’s start with something simple – a step sequencer with a sound bank and signal processing tools, built for the mobile virtual reality experience of the Samsung Gear VR.
This kind of app isn’t particularly new, and the creative tools are limited in their scope and flexibility, but the addition of a virtual reality environment (pictured left) has the potential to give the experience a more tactile immediacy that some musicians may find creatively inspiring.
“As a musician, music should always play an important role, since in VR audio makes 50% of the experience,” Sneek comments in an interview with The Creators Project. “Music and specifically creating music via software has always been a major interest of mine.”
Here’s a video demonstrating the software in action:
Carillon
“Each performer’s avatar stands on a platform within the Carillon, looking up at the giant spinning rings in the center.” Dr. Hamilton describes the Carillon during an interview with the Leap Motion Blog in which he paints a picture of how the virtual instrument looks to the musicians interacting with it inside the Oculus Rift. “Directly in front of them are a smaller set of rings with which they can interact – holding a hand over a ring or set of rings selects them, swiping gestures left/right, up/down, away/toward start the rings spinning.”
Swiping at the rings of the elaborate virtual instrument (pictured left) produces many sounds associated with the traditional Carillon, but these reverberant tones are actually generated by the Pure Data Music Engine – a programming environment used often for generative music applications. By employing the Oculus Rift and the Leap Motion device for motion tracking, Hamilton and Platz created a new, intriguing way to interact with a sonic construct assembled in Pure Data.
“The idea of a mechanical instrument that responds to performers’ motions has been something we really enjoy,” commented Dr. Hamilton. “As we all become more comfortable with technology in our daily lives, it will seem less and less strange to see it blending with traditional musical performance practice, instrument design and concert performance.”
Here’s a video showing the Carillon in a live performance. The musicians are Dr. Rob Hamilton and Stanford Assistant Professor Ge Wang:
Lyra VR
In Lyra VR, musicians and composers can manipulate a built-in library of sounds and loops to create their own compositions using a three dimensional interface within the Oculus Rift (pictured right). Unlike Carillon, which is limited to a single instrument, Lyra VR presents an array of choices and possibilities. The application was designed by Team Metanaut, led by Taiwanese VR developer Dilun Ho and an international team of developers that includes electronic musician Jean Marais (better known as MoShang). With floating buttons and many-sided geometrical shapes that glow softly in neon colors reminiscent of the world of Tron, Lyra VR is an immersive musical playground. While the instrument library may not provide a broad variety of sounds, the gestural control method could possibly afford users a sense of creative freedom and inspiration. Here’s a video of Lyra VR in action:
Music Inside the Machine
So, now we’ve explored some interesting applications that turn our VR headsets into standalone musical instruments with virtual interfaces we can manipulate with a wave of our hands, making us feel like expert VR musicians. In the third and blog post, we’ll look at how VR can reach out beyond the confines of its own applications and pull our preexisting musical instruments into the virtual world via the power of MIDI!
