Side-by-side, these are the covers of the two editions of the book. In Japanese, A Composer’s Guide to Game Music is titled “Game sound production guide – composer techniques for interactive music,” by Winifred Phillips.
I’m very excited that the Japanese language edition of my book has already hit #1 on the “Most Wished For” list on Amazon Japan!
The “Most Wished For” list on Amazon.co.jp.
Coincidentally, the English-language version of A Composer’s Guide to Game Music is now #1 on the Kindle Top Rated list, too!
The Kindle “Top Rated” list on Amazon.com.
O’Reilly Japan is located in Tokyo, and is dedicated to translating books about technological innovation for Japanese readers. They are a division of O’Reilly Media, a California publishing company that acts as “a chronicler and catalyst of leading-edge development, homing in on the technology trends that really matter and galvanizing their adoption by amplifying “faint signals” from the alpha geeks who are creating the future. O’Reilly publishes definitive books on computer technologies for developers, administrators, and users. Bestselling series include the legendary “animal books,” Missing Manuals, Hacks, and Head First.”
I’m tremendously excited about the Japanese edition of my book, and my excitement comes in large part from the venerable tradition of outstanding music in Japanese games. From the most celebrated classic scores of such top game composers as Koji Kondo (Super Mario Bros.) and Nobuo Uematsu (Final Fantasy), to the excellent modern scores of such popular composers as Masato Kouda (Monster Hunter) and Yoko Shimomura (Kingdom Hearts), Japanese video game composers have set the creative bar very high. I’m incredibly honored that my book will be read by both established and aspiring game composers in Japan! I hope they’ll find some helpful information in my book, and I’m excited to contribute to the ongoing conversation about game music in the Japanese development community.
I’ve always loved Japanese game music. In 2008, I participated in a compilation album in which successful game composers created cover versions of celebrated video game songs from classic games. The album was called “Best of the Best: A Tribute to Game Music.” I chose the music by Koji Kondo from Super Mario Bros., and recorded an a cappella vocal version. It’s currently available for sale from the Sumthing Else Music Works record label, and can also be downloaded on iTunes. You can hear the track on YouTube here:
If you’d like to learn more about the rich legacy of game music composition in Japan, you can watch an awesome free documentary series produced by the Red Bull Music Academy, entitled “Diggin’ in the Carts: A Documentary Series About Japanese Video Game Music.” The series interviews famous game composers of Japan, which means that the interviews and narration are both in Japanese (with English subtitles). Here’s an episode that focuses on modern accomplishments by Japanese game composers:
Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.
This is a continuation of my blog series on the top audio middleware options for game music composers, this time focusing on the psai Interactive Music Engine for games, developed by Periscope Studio, an audio/music production house. Initially developed as a proprietary middleware solution for use by Periscope’s in-house musicians, the software is now being made available commercially for use by game composers. In this blog I’ll take a quick look at psai and provide some tutorial resources that will further explore the utility of this audio middleware. If you’d like to read the first four blog entries in this series on middleware for the game composer, you can find them here:
The name “psai” is an acronym for “Periscope Studio Audio Intelligence,” and its lowercase appearance is intentional. Like the Elias middleware (explored in a previous installment of this blog series), the psai application attempts to provide a specialized environment specifically tailored to best suit the needs of game composers. The developers at Periscope Studio claim that psai’s “ease of use is unrivaled,” primarily because the middleware was “designed by videogame composers, who found that the approaches of conventional game audio middleware to interactive music were too complicated and not flexible enough.” The psai music engine was originally released for PC games, with a version of the software for the popular Unity engine released in January 2015.
psai graphical user interface
Both Elias and psai offer intuitive graphical user interfaces designed to ease the workflow of a game composer. However, unlike Elias, which focused exclusively on a vertical layering approach to musical interactivity, the psai middleware is structured entirely around horizontal re-sequencing, with no support for vertical layering. As I described in my book, A Composer’s Guide to Game Music, “the fundamental idea behind horizontal re-sequencing is that when composed carefully and according to certain rules, the sequence of a musical composition can be rearranged.” (Chapter 11, page 188).
Music for the psai middleware is composed in what Periscope describes as a “snippets” format, in which short chunks of music are arranged into groups that can then be triggered semi-randomly by the middleware. The overall musical composition is called a “theme,” and the snippets represent short sections of that theme. The snippets are assigned numbers that best represent degrees of emotional intensity (from most intense to most relaxed), and these intensity numbers help determine which of the snippets will be triggered at any given time. Other property assignments include whether a snippet is designated as an introductory or ending segment, or whether the snippet is bundled into a “middle” group with a particular intensity designation. Periscope cautions, “The more Middle Segments you provide, the more diversified your Theme will be. The more Middle Segments you provide for a Theme, the less repetition will occur. For a highly dynamic soundtrack make sure to provide a proper number of Segments across different levels of intensity.”
Here’s an introductory tutorial video produced by Periscope for the psai Interactive Music Engine for videogames:
Because psai only supports horizontal re-sequencing, it’s not as flexible as the more famous tools such as Wwise or FMOD, which can support projects that alternate between horizontal and vertical interactivity models. However, psai’s ease of use may prove alluring for composers who had already planned to implement a horizontal re-sequencing structure for musical interactivity. The utility of the psai middleware also seems to depend on snippets that are quite short, as is demonstrated by the above tutorial video produced by Periscope Studio. There could be some negative effects of this structure on a composer’s ability to develop melodic content (as is sometimes the case in a horizontal re-sequencing model). It would be helpful if Periscope could demonstrate psai using longer snippets that might give us a better sense of how musical ideas might be developed within the confines of their dynamic music system. One can imagine an awesome potential for creativity with this system, if the structure can be adapted to allow for more development of musical ideas over time.
And here is some gameplay video that demonstrates the music system of Mount & Blade II – Bannerlord:
Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.
Welcome back to my blog series that offers tutorial resources exploring game music middleware for the game music composer. I initially planned to write two blog entries on the most popular audio middleware solutions (Wwise and FMOD), but since I started this blog series, I’ve been hearing buzz about other middleware solution and so I thought it best to expand the series to incorporate other interesting solutions to music implementation in games. This blog will focus on a brand new middleware application called Elias, developed by Elias Software. While not as famous as Wwise or FMOD, this new application offers some intriguing new possibilities for the creation of interactive music in games.
If you’d like to read the first three blog entries in this series, you can find them here:
Elias stands for Elastic Lightweight Integrated Audio System. It is developed by Kristofer Eng and Philip Bennefall for Microsoft Windows, with a Unity plugin for consoles, mobile devices and browser-based games. What makes Elias interesting is the philosophy of its design. Instead of designing a general audio middleware tool with some music capabilities, Eng and Bennefall decided to bypass the sound design arena completely and create a middleware tool specifically outfitted for the game music composer. The middleware comes with an authoring tool called Elias Composer’s Studio that “helps the composer to structure and manage the various themes in the game and bridges the gap between the composer and level designer to ease the music integration process.”
Here’s the introductory video for Elias, produced by Elias Software:
The interactive music system of the Elias middleware application seems to favor a Vertical Layering (or vertical re-orchestration) approach with a potentially huge number of music layers able to play in lots of combinations. The system includes flexible options for layer triggering, including the ability to randomize the activation of the layers to keep the listening experience unpredictable during gameplay.
As a middleware application designed specifically to address the top needs of game music composers, Elias is certainly intriguing! The software has so far been used in only one published game – Gauntlet, which is the latest entry in the awesome video game franchise first developed by Atari Games for arcade cabinets in 1985. This newest entry in the franchise was developed by Arrowhead Game Studios for Windows PCs. We can hear the Elias middleware solution in action in this gameplay video from Gauntlet:
Elias Software recently demonstrated its Elias middleware application on the expo floor of the Nordic Game 2015 conference in Malmö, Sweden (May 20-22, 2015). Here’s a look at Elias’ booth from the expo:
Since Elias is a brand new application, I’ll be curious to see how widely it is accepted by the game audio community. A middleware solution that focuses solely on music is definitely a unique approach! If audio directors and audio programmers embrace Elias, then it may have the potential to give composers better tools and an easier workflow in the creation of interactive music for games.
At least, that was my impression a couple of months ago when I attended the audio track of the Game Developers Conference. Setting a new record for attendance, GDC hosted over 24,000 game industry pros who flocked to San Francisco’s Moscone Center in March for a full week of presentations, tutorials, panels, awards shows, press conferences and a vibrant exposition floor filled with new tech and new ideas. As one of those 24,000 attendees, I enjoyed meeting up with lots of my fellow game audio folks, and I paid special attention to the presentations focusing on game audio. Amongst the tech talks and post-mortems, I noticed a lot of buzz about a subject that used to be labeled as very old-school: MIDI.
This was particularly emphasized by all the excitement surrounding the new MIDI capabilities in the Wwise middleware. In October of 2014, Wwise released its most recent version (2014.1) which introduced a number of enhanced features, including “MIDI support for interactive music and virtual instruments (Sampler and Synth).” Wwise now allows the incorporation of MIDI that triggers either a built-in sound library in Wwise or a user-created one. Since I talk about the future of MIDI game music in my book, A Composer’s Guide to Game Music, and since this has become a subject of such avid interest in our community, I thought I’d do some research on this newest version of Wwise and post a few resources that could come in handy for any of us interested in embarking in a MIDI game music project using Wwise 2014.1.
The first is a video produced by Damian Kastbauer, technical audio lead at PopCap games and the producer and host of the now-famous Game Audio Podcast series. This video was released in April of 2014, and included a preview of the then-forthcoming MIDI and synthesizer features of the new Wwise middleware tool. In this video, Damian takes us through the newest version of the “Project Adventure” tutorial prepared by Audiokinetic, makers of Wwise. In the process, he gives us a great, user-friendly introduction to the MIDI capabilities of Wwise.
The next videos were produced by Berrak Nil Boya, a composer and contributing editor to the Designing Sound website. In these videos, Berrak has taken us through some of the more advanced applications of the MIDI capabilities of Wwise, starting with the procedure for routing MIDI data directly into Wwise from more traditional MIDI sequencer software such as that found in a Digital Audio Workstation (DAW) application. This process would allow a composer to work within more traditional music software and then directly route the MIDI output into Wwise. Berrak takes us through the process in this two-part video tutorial:
Finally, Berrak Nil Boya has created a video tutorial on the integration of Wwise into Unity 5, using MIDI. Her explanation of the preparation of a soundbank and the association of MIDI note events with game events is very interesting, and provides a nicely practical application of the MIDI capability of Wwise.
Welcome back to my series of blogs that collect some tutorial resources about game music middleware for the game music composer. I had initially intended to publish two blog entries on this subject, focusing on the most popular audio middleware solutions: Wwise and FMOD. However, since the Fabric audio middleware has been making such a splash in the game audio community, I thought I’d extend this series to include it. If you’d like to read the first two blog entries in this series, you can find them here:
Fabric is developed by Tazman Audio for the Unity game engine (which enables game development for consoles, PCs, mobile devices such as iOS and Android, and games designed to run within a web browser). Here’s a Unity game engine overview produced by Unity Technologies:
The Fabric middleware is designed to expand the audio capabilities of the Unity game engine. The complete product manual for the Fabric middleware is available online. The video tutorials that I’m featuring below were created by two game audio professionals who have very generously walked us through the use of the software. If you’d like a more nuts-and-bolts overview of the software features of Fabric, you can find it here.
The first video was shot in 2013 during the Konsoll game development conference in Norway, and gives an overview of the general use of Fabric in game audio. The speaker, Jory Prum, is an accomplished game audio professional whose game credits include The Walking Dead, The Wolf Among Us, Broken Age, SimCity 4, Star Wars: Knights of the Old Republic, and many more.
Making a great sounding Unity game using Fabric
In the next two-part video tutorial, composer Anastasia Devana has expanded on her previous instructional videos about FMOD Studio, focusing now on recreating the same music implementation strategies and techniques using the Fabric middleware in Unity. Anastasia Devana is an award-winning composer whose game credits include the recently released puzzle game Synergy and the upcoming roleplaying game Anima – Gate of Memories.
Fabric and Unity: Adaptive Music in Angry Bots – Part 1
Fabric and Unity: Adaptive Music in Angry Bots – Part 2
Classic FM is the only 100% classical music radio station in the UK. Every year, they hold a poll to select Britain’s favorite pieces of classical music, listing the top 300 selections on their web site. Over 100,000 people voted in this year’s poll. The final results include music from video games – 11 out of the 300 compositions are pieces of video game music, including three pieces that won places in the top 20. To celebrate, I’ve gathered together some YouTube videos presenting the famous video game music that was voted into the top 300 in Classic FM’s poll. I hope you enjoy it!
#9. Final Fantasy Series (Nobuo Uematsu)
#11. The Elder Scrolls Series (Jeremy Soule)
#13. Banjo-Kazooie (Grant Kirkhope)
#30. Kingdom Hearts (Yoko Shimomura)
#41. Viva Piñata (Grant Kirkhope)
#53. World of Warcraft (Russell Brower, Neal Acree, Jason Hayes, Tracy Bush, et. al.)
#59. Kingdoms of Amalur: Reckoning (Grant Kirkhope)
Ready or not, virtual reality is coming! Three virtual reality headsets are on their way to market and expected to hit retail in either late 2015 or sometime in 2016. These virtual reality systems are:
VR is expected to make a big splash in the gaming industry, with many studios already well underway with development of games that support the new VR experience. Clearly, VR will have a profound impact on the visual side of game development, and certainly sound design and voice performances will be impacted by the demands of such an immersive experience… but what about music? How does music fit into VR?
At GDC 2015, a presentation entitled “Environmental Audio and Processing for VR” laid out the technology of audio design and implementation for Sony’s Project Morpheus system. While the talk concentrated mainly on sound design concerns, speaker Nicholas Ward-Foxton (audio programmer for Sony Computer Entertainment) touched upon voice-over and music issues as well. Let’s explore his excellent discussion of audio implementation for a virtual space, and ponder how music fits into this brave new virtual world.
Nicholas Ward-Foxton, during his GDC 2015 talk.
But first, let’s get a brief overview on audio in VR:
3D Positional Audio
All three VR systems feature some sort of positional audio, meant to achieve a full 3D Audio Effect. With the application of the principles of 3D Audio, sounds will always seem to be originating from the virtual world in a realistic way, according to the location of the sound-creating object, the force/loudness of the sound being emitted, the acoustic character of the space in which the sound is occurring, and the influences of obstructing, reflecting and absorbing objects in the surrounding environment. The goal is to create a soundscape that seems perfectly fused with the visual reality presented to the player. Everything the player hears seems to issue from the virtual world with acoustic qualities that consistently confirm an atmosphere of perfect realism.
To get a greater appreciation of the power of 3D audio, let’s listen to the famous “Virtual Barber Shop” audio illusion, created by QSound Labs to demonstrate the power of Binaural audio.
Head Tracking and Head-Related Transfer Function
According to Nicholas Ward-Foxton’s GDC talk, to make the three-dimensional audio more powerful in a virtual space, the VR systems need to keep track of the player’s head movements and adjust the audio positioning accordingly. With this kind of head tracking, sounds swing around the player when turning or looking about. This effect helps to offset an issue of concern in regards to the differences in head size and ear placement between individuals. In short, people have differently sized noggins, and their perception of audio (including the 3D positioning of sounds) will differ as a result. This dependance on the unique anatomical details of the individual listener is known as Head-Related Transfer Function. There’s an excellent article explaining Head-Related Transfer Function on the “How Stuff Works” site.
Head-Related Transfer Function can complicate things when trying to create a convincing three-dimensional soundscape. When listening to identical binaural audio content, one person may not interpret aural signals the same way another would, and might estimate that sounds are positioned differently. Fortunately, head tracking comes to the rescue here. As Ward-Foxton explained during his talk, when we move our heads about and then listen to the way that the sounds shift in relation to our movements, our brains are able to adjust to any differences in the way that sounds are reaching us, and our estimation of the spatial origination of individual sounds becomes much more reliable. So the personal agency of the gaming experience is a critical element in completing the immersive aural world.
Music, Narration, and the Voice of God
Now, here’s where we start talking about problems relating directly to music in a VR game. Nicholas Ward-Foxton’s talk touched briefly on the issues facing music in VR by exploring the two classifications that music may fall into. When we’re playing a typical video game, we usually encounter both diegetic and non-diegetic audio content. Diegetic audio consists of sound elements that are happening in the fictional world of the game, such as environment sounds, sound effects, and music being emitted by in-game sources such as radios, public address systems, NPC musicians, etc. On the other hand, non-diegetic audio consists of sound elements that we understand to be outside the world of the story and its characters, such as a voice-over narration, or the game’s musical score. We know that the game characters can’t hear these things, but it doesn’t bother us that we can hear them. That’s just a part of the narrative.
VR changes all that. When we hear a disembodied, floating voice from within a virtual environment, we sometimes feel, according to Ward-Foxton, as though we are hearing the voice of God. Likewise, when we hear music in a VR game, we may sometimes perceive it as though it were God’s underscore. I wrote about the problems of music breaking immersion as it related to mixing game music in surround sound in Chapter 13 of my book, A Composer’s Guide to Game Music, but the problem becomes even more pronounced in VR.When an entire game is urging us to suspend our disbelief fully and become completely immersed, the sudden intrusion of the voice of the Almighty supported by the beautiful strains of the holy symphony orchestra has the potential to be pretty disruptive.
The harpist of the Almighty, hovering somewhere in the VR world…
So, what can we do about it? For non-diegetic narration, Ward-Foxton suggested that the voice would have to be contextualized within the in-game narrative in order for the “voice of God” effect to be averted. In other words, the narration needs to come from some explainable in-game source, such as a radio, a telephone, or some other logical sound conveyance that exists in the virtual world. That solution, however, doesn’t work for music, so it’s time to start thinking outside the box.
Voice in our heads
During the Q&A portion of Ward-Foxton’s talk, an audience member asked a very interesting question. When the player is assuming the role of a specific character in the game, and that character speaks, how can the audio system make the resulting spoken voice sound the way it would to the ears of the speaker? After all, whenever any of us speak aloud, we don’t hear our voices the way others do. Instead, we hear our own voice through the resonant medium of our bodies, rising from our larynx and reverberating throughout our own unique formant, or acoustical vocal tract. That’s why most of us perceive our voices as being deeper and richer than they sound when we hear them in a recording.
Ward-Foxton suggested that processing and pitch alteration might create the effect of a lower, deeper voice, helping to make the sound seem more internal and resonant (the way it would sound to the actual speaker). However, he also mentioned another approach to this issue earlier in his talk, and I think this particular approach might be an interesting solution for the “music of God” problem as well.
Proximity Effect
“I wanted to talk about proximity,” said Ward-Foxton, “because it’s a really powerful effect in VR, especially audio-wise.” Referencing the Virtual Barber Shop audio demo from QSound Labs, Ward-Foxton talked about the power of sounds that seem to be happening “right in your personal space.” In order to give sounds that intensely intimate feeling when they become very close, Ward-Foxton’s team would apply dynamic compression and bass boost to the sounds, in order to simulate the Proximity Effect.
The Proximity Effect is a phenomenon related to the physical construction of microphones, making them prone to add extra bass and richness when the source of the recording draws very close to the recording apparatus. This concept is demonstrated and explained in much more depth in this video produced by Dr. Alexander J. Turner for the blog Nerds Central:
So, if simulating the Proximity Effect can make a voice sound like it’s coming from within, as Ward-Foxton suggests, can applying some of the principles of the Proximity Effect make the music sound like it’s coming from within, too?
Music in our heads
This was the thought that crossed my mind during this part of Ward-Foxton’s talk on “Environmental Audio and Processing for VR.” In traditional music recording, instruments are assigned a position on the stereo spectrum, and the breadth from left to right can feel quite wide. Meanwhile, the instruments (especially in orchestral recordings) are often recorded in an acoustic space that would be described as “live,” or reverberant to some degree. This natural reverberance is widely regarded as desirable for an acoustic or orchestral recording, since it creates a sensation of natural space and allows the sounds of the instruments to blend with the assistance of the sonic reflections from the recording environment. However, it also creates a sensation of distance between the listener and the musicians. The music doesn’t seem to be invading our personal space. It’s set back from us, and the musicians are also spread out around us in a large arc shape.
So, in VR, these musicians would be invisibly hovering in the distance, their sounds emitting from defined positions in the stereo spectrum. Moreover the invisible musicians would fly around as we turn our heads, maintaining their position in relation to our ears, even as the sound design elements of the in-game environment remain consistently true to their places of origin in the VR world. Essentially, we’re listening to the Almighty’s holy symphony orchestra. So, how can we fix this?
One possible approach might be to record our music with a much more intimate feel. Instead of choosing reverberant spaces, we might record in perfectly neutral spaces and then add very subtle amounts of room reflection to assist in a proper blend without disrupting the sensation of intimacy. Likewise, we might somewhat limit the stereo positioning of our instruments, moving them a bit more towards the center. Finally, a bit of prudently applied compression and EQ might add the extra warmth and intimacy needed in order to make the music feel close and personal. Now, the music isn’t “out there” in the game world. Now, the music is in our heads.
Music in VR
It will be interesting to see the audio experimentation that is surely to take place in the first wave of VR games. So far, we’ve only been privy to tech demos showing the power of the VR systems, but the music in these tech demos has given us a brief peek at what music in VR might be like in the future. So far, it’s been fairly sparse and subtle… possibly a response to the “music of the Almighty” problem. It is interesting to see how this music interacts with the gameplay experience. Ward-Foxton mentioned two particular tech demos during his talk. Here’s the first, called “Street Luge.”
The simple music of this demo, while quite sparse, does include some deep, bassy tones and some dry, close-recorded percussion. Also, the stereo breadth appears to be a bit narrow as well, but this may not have been intentional.
The second tech demo mentioned during Ward-Foxton’s talk was “The Deep.”
The music of this tech demo is limited to a few atmospheric synth tones and a couple of jump-scare stingers, underscored by a deep low pulse. Again, the music doesn’t seem to have a particularly wide stereo spectrum, but this may not have been a deliberate choice.
I hope you enjoyed this exploration of some of the concepts included in Nicholas Ward-Foxton’s talk at GDC 2015, along with my own speculation about possible approaches to problems related to non-diegetic music in virtual reality. Please let me know what you think in the comments!
At this year’s Game Developers Conference, voice director Michael Csurics presented a terrific talk called “VO Session Live: Ensemble Edition.” By soliciting the audience for volunteer voice actors, Csurics staged a live voice-over recording session that included both solo actor/performers and multiple actors running dialogue in a traditionally theatrical way. The demonstration served to reveal the ins-and-outs of recording voice dialogue, both from an artistic and technical standpoint. One of the portions of Csurics’ talk that I found particularly interesting was his exploration of the process of multiple takes in recording voice dialogue, and I thought his method might have some bearing on the process of recording multiple takes of a musician’s performance. In this blog, I’ll be going over the basics of his methodology, as he described it during his talk, and I’ll also be sharing some of my thoughts regarding how his working method intersects with my experiences recording session performances with live musicians.
Michael Csurics giving his talk, “VO Session Live: Ensemble Edition”
For typical voice-over recording sessions, Csurics begins by having his voice performers record a short piece of dialogue at least two or three times. This process “gets them comfortable, gets them past the cold read and into an actual read.” During these voice sessions he will be assisted by an audio engineer running the Pro Tools session. Csurics will usually pause after the first two or three readings, sometimes giving direction and asking for additional takes. Once satisfied, he will then tell the audio engineer which take he liked, and the audio engineer will “pull down” this take. In other words, the engineer will select the portion of the recorded waveform that represents the final take and copy/paste it into a blank audio track directly below the track that is currently recording the vocal session. In this way, Csurics is able to save post-production time by making his final selections on-the-fly.
Michael Csurics’ isolation booth for vocal recording
When I’m recording live musicians (either my own performances or those of others), my own workflow resembles Csurics’ process in some respects. I’ll have the backing music mix loaded into its own audio tracks in Pro Tools, and then I’ll set up a collection of blank audio tracks below the rest of the music mix. The number of blank tracks will largely depend on how many takes I expect will be required. For especially difficult parts in which a large number of takes are anticipated, I may set up anywhere from four to fifteen blank tracks. The musician (or myself, if that’s the case) will record a short section of the music, repeating that section for as many takes as may seem warranted. Each take will be recorded into one of the separate audio tracks that I’d set up prior to the recording session. Once recorded, I’ll mute that audio track, record-enable the next one and then record the next take. Once complete, I’ll have a fairly large assortment of takes that I can audition during post-production in order to edit together a version that will be used in the final recording. During post, I employ the same “pull down” method that Csurics described during his talk – i.e. copy/pasting the best version of each performance into a blank audio track at the bottom of my Pro Tools edit window.
I admire Csurics’ on-the-fly method of selecting the best take, but personally I’m only comfortable with making an instant decision if it pertains to simply discarding a take completely and trying again. In this case, I’ll delete the recording from the edit window, ensuring that I avoid any possibility of confusion later. Using this method for recording live instrumental performances gives me a large amount of flexibility in post, allowing me to assemble the most ideal musical performance possible. I can listen critically to every phrase in every take, selecting only the very best execution of every musical element for inclusion in the final recording. I talk more about some of the technical aspects of my workflow in my book, A Composer’s Guide to Game Music.
The Bleeps ‘n’ Bloops game audio podcast
There were lots more details regarding voice dialogue tracking and editing in Csurics’ talk, which is available in the GDC Vault. If you don’t have access to the Vault, you can hear a lot of those same ideas in this great podcast interview with Csurics, recorded by the Bleeps ‘n’ Bloops podcast during last year’s GameSoundCon. The interview includes some interesting details about workflow, track naming, post editing and processing. I’ve embedded the SoundCloud recording of that podcast interview below. The section regarding voice-over session logistics starts at 11:40:
There’s also an earlier interview with Csurics posted on the Game Audio Network Guild site, which reflects on some of Csurics’ experiences as the Dialogue Supervisor for 2K Marin:
Since leaving 2K Games in 2012, Michael Csurics has worked as an independent contractor and now runs a game audio production services company called The Brightskull Entertainment Group (he can be reached on Twitter at @mcsurics). If you’d like to see and hear the great work going on at Brightskull, here’s a video demonstration from one of their latest projects, The Vanishing of Ethan Carter:
In a previous blog post, we took a look at a few tutorial resources for the latest version of the Wwise audio middleware. One of the newest innovations in the Wwise software package is a fairly robust MIDI system. This system affords music creators and implementers the opportunity to avail themselves of the extensive adaptive possibilities of the MIDI format from within the Wwise application. Last month, during the Game Developers Conference in the Moscone Center in San Francisco, some members of the PopCap audio development team presented a thorough, step-by-step explanation of the benefits of this MIDI capability for one of their latest projects, Peggle Blast. Since my talk during the Audio Bootcamp at GDC focused on interactive music and MIDI (with an eye on the role of MIDI in both the history and future of game audio development), I thought that we could all benefit from a summation of some of the ideas discussed during the Peggle Blast talk, particularly as they relate to dynamic MIDI music in Wwise. In this blog, I’ve tried to convey some of the most important takeaways from this GDC presentation.
“Peggle Blast: Big Concepts, Small Project” was presented on Thursday, March 5th by three members of the PopCap audio team: technical sound designer RJ Mattingly, audio lead Jaclyn Shumate, and senior audio director Guy Whitmore. The presentation began with a quote from Igor Stravinsky:
The more constraints one imposes, the more one frees oneself, and the arbitrariness of the constraint serves only to maintain the precision of the execution.
This idea became a running theme throughout the presentation, as the three audio pros detailed the constraints under which they worked, including:
A 5mb memory limit for all audio assets
Limited CPU
2.5mb memory allocation for the music elements
These constraints were a result of the mobile platforms (iOS and Android) for which Peggle Blast had been built. For this reason, the music team focused their attention on sounds that could convey lots of emotion while also maintaining a very small file size. Early experimentation with tracks structured around the use of a music box instrument led the team to realize that they still needed to replicate the musical experience from the full-fledged console versions of the game. A simple music-box score was too unsatisfying, particularly for players who were familiar with the music from the previous installments in the franchise. With that in mind, the team concentrated on very short orchestral samples taken from the previous orchestral session recordings for Peggle 2. Let’s take a look at a video from those orchestral sessions:
Using those orchestral session recordings, the audio team created custom sample banks that were tailored specifically to the needs of Peggle Blast, focusing on lots of very short instrument articulations and performance techniques including:
pizzicato
marcato
staccato
mallets
A few instruments (including a synth pad and some orchestral strings) were edited to loop so that extended note performances became possible, but the large majority of instruments remained brief, punctuated sounds that did not loop. These short sounds were arranged into sample banks in which one or two note samples would be used per octave of instrument range, and note tracking would transpose the sample to fill in the rest of the octave. The sample banks consisted of a single layer of sound, which meant that the instruments did not adjust their character depending on dynamics/velocity. In order to make the samples more musically pleasing, the built-in digital signal processing capability of Wwise was employed by way of a real-time reverb bus that allowed these short sounds to have more extended and natural-sounding decay times.
The audio team worked with a beta version of Wwise 2014 during development of Peggle Blast, which allowed them to implement their MIDI score into the Unity game engine. The composer, Guy Whitmore, composed the music in a style consisting of whimsically pleasant, non-melodic patterns that were structured into a series of chunks. These chunks could be triggered according to the adaptive system in Peggle Blast, wherein the music went through key changes (invariably following the circle of fifths) in reaction to the player’s progress. To better see how this works, let’s watch an example of some gameplay from Peggle Blast:
As you can see, very little in the way of a foreground melody existed in this game. In the place of a melody, foreground musical tones would be emitted when the Peggle ball hit pegs during its descent from the top of the screen. These tones would follow a predetermined scale, and would choose which type of scale to trigger (major, natural minor, harmonic minor, or mixolydian) depending on the key in which the music was currently playing. Information about the key was dropped into the music using markers that indicated where key changes took place, so that the Peggle ball would always trigger the correct type of scale at any given time. The MIDI system did not have to store unique MIDI data for scales in every key change, but would instead calculate the key transpositions for each of the scale types, based on the current key of the music that was playing.
The presentation ended with an emphasis on the memory savings and flexibility afforded by MIDI, and the advantages that MIDI presents to game composers and audio teams. It was a very interesting presentation! If you have access to the GDC Vault, you can watch a video of the entire presentation online. Otherwise, there are plenty of other resources on the music of Peggle Blast, and I’ve included a few below:
Since third-party audio middleware in game development is becoming slowly more prevalent, I’m devoting two blog entries to some tutorials by game audio pros who have produced videos to demonstrate their process working with the software. I posted the first blog entry in February — you can read it here.
This second blog is devoted to FMOD, and the tutorials were produced by two game composers who have generously shared their experiences. The first video focuses on the creation of adaptive music for a demo competition hosted by the Game Audio Network Guild and taking place during the Game Developers Conference 2014 in San Francisco. The tutorial was produced by composer Anastasia Devana, whose game credits include the recently released puzzle game Synergy and the upcoming roleplaying game Anima – Gate of Memories.
Adaptive music in Angry Bots using FMOD Studio and Unity
The next tutorials come to us from composer Matthew Pablo, who produced a series of videos on the implementation of game music via the FMOD middleware. Matthew’s work as a game composer includes N-Dimensions, Arizona Rose and the Pharaoh’s Riddles, Cloud Spin, Micromon, and many more. Here is the first video — the rest can be found in Matthew’s YouTube playlist.
Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution. Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games. Follow her on Twitter @winphillips.
