Hello there! I’m video game composer Winifred Phillips. Next week, I’ll be giving a lecture during the Game Developers Conference 2021 event. During my lecture, I’ll be talking about the music I composed for Sumo Digital for both the Sackboy: A Big Adventure and Spyder video games. My lecture is entitled, “From Spyder to Sackboy: A Big Adventure in Interactive Music,” and will take place on Friday July 23rd at 3:40pm PT. Although GDC is still an all-virtual affair, the event does provide lots of opportunities for experts within the game development community to share their knowledge, coupled with forums enabling game audio folks to network and learn from each other. In addition to my prepared lecture, I’ll also be participating in a live Speaker Q&A that will take place right after my presentation. It should be a lot of fun! Really looking forward to sharing my experience working with Sumo Digital simultaneously on these two fantastic games.
This was an incredibly rare and awesome opportunity for me to compose music for two projects simultaneously in development by the same company. Because of this, I found the comparisons between the two games fascinating.
My talk will delve into the mechanics of the dynamic music systems in both games, showing how a comparison between these two projects can shed some light on the utility of the top interactive techniques and strategies. While comparing this list of interactive music techniques provided me with a lot of material for my GDC lecture, there were other ways in which the two projects were similar. I thought I’d share some brief thoughts on one of the other common threads I found between these two Sumo Digital games.
As composers, we’re often asked to provide a general atmosphere that adds either character to gameplay or distinctive flavor to menus. If it’s a horror game, maybe we’re being asked to provide a crushingly heavy drone of doom during tense exploration, with soul-shuddering tone clusters bubbling up from the darkness and then sinking back down into the murky depths. For a whimsical game, we might be creating airy, open textures with little mischievous accents from the mallets or woodwind section… or maybe we’re creating a brightly whimsical melody for an opening menu or splash screen. If it’s a fantasy roleplaying game, we may be providing softly ambient tracks for exploration, with a pensive flute wandering gently through Gaelic figures. Or maybe we’re creating a thunderously epic main theme for an opening menu, designed to emphasize the world-shattering stakes of the adventure to come.
So happy you’ve joined us! I’m videogame composer Winifred Phillips. Welcome to my two-part article series on the process of composing music for timed challenges in video games! Since timed challenges are a popular gameplay mechanic that has featured prominently in my most recently released project (The Spyder DLC missions), I thought it might be interesting for us to take a closer look at what makes a timed challenge tick!
Welcome! I’m video game composer Winifred Phillips, and in this month’s article, I’d like to go into some depth about an interesting aspect of our work as game composers – creating music for timed challenges. In timed challenges, players must complete a set of tasks within a limited window of time. Over the years, I’ve created music for lots of timed challenges featured in highly divergent projects, from the darkly strategic space battles of Hades’ Star (pictured above), to the wacky assembly-line mayhem of the Fail Factory VR game, to the brand-new DLC release from one of my most recent projects — the Spyder video game. It was this most recent release that actually got me thinking a lot about how difficult timed challenges can be for game composers.
Welcome! I’m video game composer Winifred Phillips, and I’m glad you’ve joined us for this continuation of our discussion of the dynamic music system in the video game Spyder! As you may recall from our previous discussion, Spyder is a spy thriller set in a retro world that’s vibrant with the famously over-the-top music and aesthetic of the late 1960s to early 1970s. The game was developed by Sumo Digital for the popular Apple Arcade gaming platform. The protagonist is an intelligent gadget resembling a tiny robotic spider. This device, named “Agent 8,” was created by an elite British spy organization. As the hero of the game, Agent 8 undertakes high-stakes espionage in order to defeat a sprawling evil organization known as S.I.N.! Sumo Digital recently released a developer diary video about the making of the music of SPYDER, so let’s check that out:
As you could see from the video, the Spyder video game features a dynamic music system designed to convey the iconic 1960s style of a classic spy thriller. In this two-part article series, we’ve been exploring how that system was created.
Hello there! I’m video game composer Winifred Phillips, and I’m excited to announce the release of my most recent video game project – Spyder, developed by Sumo Digital for the popular Apple Arcade gaming platform. I loved working with the amazing audio team at Sumo Digital, and composing the music of Spyder was an absolute blast! As a retro spy thriller with a really iconic visual aesthetic, Spyder gave me the chance to delve into the musical styles of the late sixties and early seventies. Big band jazz of the 50s had evolved over time into a groovy psychedelic circus of 1960s musical fun. Mix this with the beginnings of 70s funk – and early synthetic sounds such as the famous Minimoog – and you end up with a potent cocktail of musical influences and attitudes. All of this retro goodness is reflected in the old-school movie-style poster created by the Sumo Digital team to announce the Spyder video game (pictured right).
The historical research into style, technique and instrumentation posed a significant challenge for me as a game music composer. In the course of preparing to compose the music for Spyder, I sank an enormous amount of time into this research, listening to what felt like every single spy movie soundtrack from the late sixties and early seventies. I also listened to tons of straight action movie soundtracks from the same era, as well as a great assortment of comedies, all while taking copious notes. Lending a strong sense of authenticity to the era was a crucial responsibility of the game music that would give Spyder its evocative character.
Welcome back to my three-part blog series on the art of arrangement for dynamic music systems in games! In this series of articles, I’m discussing the techniques of arrangement as they pertain to interactive game music by exploring examples from the music I composed for video games from the LittleBigPlanet franchise. In part one of this series, we went over the role of the arranger, the importance of an interesting and creative arrangement, and the relationship between arranging for traditional linear and non-linear interactive music. We also reviewed arranging techniques that apply to melody, and how these should (or should not) be applied in an interactive composition. If you haven’t read part one, please click here to read that entry first, and then return here to continue reading part two. Okay, are you back now? Ready? Here we go!
Welcome back to my blog series that offers tutorial resources exploring game music middleware for the game music composer. I initially planned to write two blog entries on the most popular audio middleware solutions (Wwise and FMOD), but since I started this blog series, I’ve been hearing buzz about other middleware solution and so I thought it best to expand the series to incorporate other interesting solutions to music implementation in games. This blog will focus on a brand new middleware application called Elias, developed by Elias Software. While not as famous as Wwise or FMOD, this new application offers some intriguing new possibilities for the creation of interactive music in games.
If you’d like to read the first three blog entries in this series, you can find them here:
Elias stands for Elastic Lightweight Integrated Audio System. It is developed by Kristofer Eng and Philip Bennefall for Microsoft Windows, with a Unity plugin for consoles, mobile devices and browser-based games. What makes Elias interesting is the philosophy of its design. Instead of designing a general audio middleware tool with some music capabilities, Eng and Bennefall decided to bypass the sound design arena completely and create a middleware tool specifically outfitted for the game music composer. The middleware comes with an authoring tool called Elias Composer’s Studio that “helps the composer to structure and manage the various themes in the game and bridges the gap between the composer and level designer to ease the music integration process.”
Here’s the introductory video for Elias, produced by Elias Software:
The interactive music system of the Elias middleware application seems to favor a Vertical Layering (or vertical re-orchestration) approach with a potentially huge number of music layers able to play in lots of combinations. The system includes flexible options for layer triggering, including the ability to randomize the activation of the layers to keep the listening experience unpredictable during gameplay.
As a middleware application designed specifically to address the top needs of game music composers, Elias is certainly intriguing! The software has so far been used in only one published game – Gauntlet, which is the latest entry in the awesome video game franchise first developed by Atari Games for arcade cabinets in 1985. This newest entry in the franchise was developed by Arrowhead Game Studios for Windows PCs. We can hear the Elias middleware solution in action in this gameplay video from Gauntlet:
Elias Software recently demonstrated its Elias middleware application on the expo floor of the Nordic Game 2015 conference in Malmö, Sweden (May 20-22, 2015). Here’s a look at Elias’ booth from the expo:
Since Elias is a brand new application, I’ll be curious to see how widely it is accepted by the game audio community. A middleware solution that focuses solely on music is definitely a unique approach! If audio directors and audio programmers embrace Elias, then it may have the potential to give composers better tools and an easier workflow in the creation of interactive music for games.
In a previous blog post, we took a look at a few tutorial resources for the latest version of the Wwise audio middleware. One of the newest innovations in the Wwise software package is a fairly robust MIDI system. This system affords music creators and implementers the opportunity to avail themselves of the extensive adaptive possibilities of the MIDI format from within the Wwise application. Last month, during the Game Developers Conference in the Moscone Center in San Francisco, some members of the PopCap audio development team presented a thorough, step-by-step explanation of the benefits of this MIDI capability for one of their latest projects, Peggle Blast. Since my talk during the Audio Bootcamp at GDC focused on interactive music and MIDI (with an eye on the role of MIDI in both the history and future of game audio development), I thought that we could all benefit from a summation of some of the ideas discussed during the Peggle Blast talk, particularly as they relate to dynamic MIDI music in Wwise. In this blog, I’ve tried to convey some of the most important takeaways from this GDC presentation.
“Peggle Blast: Big Concepts, Small Project” was presented on Thursday, March 5th by three members of the PopCap audio team: technical sound designer RJ Mattingly, audio lead Jaclyn Shumate, and senior audio director Guy Whitmore. The presentation began with a quote from Igor Stravinsky:
The more constraints one imposes, the more one frees oneself, and the arbitrariness of the constraint serves only to maintain the precision of the execution.
This idea became a running theme throughout the presentation, as the three audio pros detailed the constraints under which they worked, including:
A 5mb memory limit for all audio assets
2.5mb memory allocation for the music elements
These constraints were a result of the mobile platforms (iOS and Android) for which Peggle Blast had been built. For this reason, the music team focused their attention on sounds that could convey lots of emotion while also maintaining a very small file size. Early experimentation with tracks structured around the use of a music box instrument led the team to realize that they still needed to replicate the musical experience from the full-fledged console versions of the game. A simple music-box score was too unsatisfying, particularly for players who were familiar with the music from the previous installments in the franchise. With that in mind, the team concentrated on very short orchestral samples taken from the previous orchestral session recordings for Peggle 2. Let’s take a look at a video from those orchestral sessions:
Using those orchestral session recordings, the audio team created custom sample banks that were tailored specifically to the needs of Peggle Blast, focusing on lots of very short instrument articulations and performance techniques including:
A few instruments (including a synth pad and some orchestral strings) were edited to loop so that extended note performances became possible, but the large majority of instruments remained brief, punctuated sounds that did not loop. These short sounds were arranged into sample banks in which one or two note samples would be used per octave of instrument range, and note tracking would transpose the sample to fill in the rest of the octave. The sample banks consisted of a single layer of sound, which meant that the instruments did not adjust their character depending on dynamics/velocity. In order to make the samples more musically pleasing, the built-in digital signal processing capability of Wwise was employed by way of a real-time reverb bus that allowed these short sounds to have more extended and natural-sounding decay times.
The audio team worked with a beta version of Wwise 2014 during development of Peggle Blast, which allowed them to implement their MIDI score into the Unity game engine. The composer, Guy Whitmore, composed the music in a style consisting of whimsically pleasant, non-melodic patterns that were structured into a series of chunks. These chunks could be triggered according to the adaptive system in Peggle Blast, wherein the music went through key changes (invariably following the circle of fifths) in reaction to the player’s progress. To better see how this works, let’s watch an example of some gameplay from Peggle Blast:
As you can see, very little in the way of a foreground melody existed in this game. In the place of a melody, foreground musical tones would be emitted when the Peggle ball hit pegs during its descent from the top of the screen. These tones would follow a predetermined scale, and would choose which type of scale to trigger (major, natural minor, harmonic minor, or mixolydian) depending on the key in which the music was currently playing. Information about the key was dropped into the music using markers that indicated where key changes took place, so that the Peggle ball would always trigger the correct type of scale at any given time. The MIDI system did not have to store unique MIDI data for scales in every key change, but would instead calculate the key transpositions for each of the scale types, based on the current key of the music that was playing.
The presentation ended with an emphasis on the memory savings and flexibility afforded by MIDI, and the advantages that MIDI presents to game composers and audio teams. It was a very interesting presentation! If you have access to the GDC Vault, you can watch a video of the entire presentation online. Otherwise, there are plenty of other resources on the music of Peggle Blast, and I’ve included a few below:
LittleBigPlanet 3 and Beyond: Taking Your Score to Vertical Extremes
I was honored to be selected by the Game Developers Conference Advisory Board to present two talks during this year’s GDC in San Francisco earlier this month. On Friday March 6th I presented a talk on the music system of the LittleBigPlanet franchise. Entitled “LittleBigPlanet 3 and Beyond: Taking Your Score to Vertical Extremes,” the talk explored the Vertical Layering music system that has been employed in all of the LittleBigPlanet games (the soundtrack for that game is available here). I’ve been on the LittleBigPlanet music composition team for six of their games so far, and my talk used many examples from musical compositions I created for all six of those projects.
After my talk, several audience members let me know that the section of my presentation covering the music system for the Pod menu of LittleBigPlanet 3 was particularly interesting – so I thought I’d share the concepts and examples from that part of my presentation in this blog.
That’s me, giving my GDC speech on the interactive music system of the LittleBigPlanet franchise. Here I’m just starting the section about the Pod menu music.
The audio team at Media Molecule conceived the dynamic music system for the LittleBigPlanet franchise. According to the franchise’s music design brief, all interactive tracks in LittleBigPlanet games must be arranged in a vertical layering system. I discussed this type of interactive music in a blog I published last year, but I’ll recap the system briefly here as well. In a vertical layering music system, the music is not captured in a single audio recording. Instead, several audio recordings play in sync with one other. Each layer of musical sound features unique content. Each of the layers represents a certain percentage of the entire musical composition. Played all together, we hear the full mix embodying the entire musical composition. Played separately, we hear submixes that are still satisfying and entertaining for their own sake. The music system can play all the layers either together or separately, or can combine the layers into different sets that represent a portion of the whole mix.
When implemented into gameplay, layers are often activated when the player moves into a new area. This helps the music to feel responsive to the player’s actions. The music seems to acknowledge the player’s progress throughout the game. It’s important to think about the way in which individual layers may be activated, and the functions that the layers may be called upon to serve during the course of the game.
In LittleBigPlanet 3, the initial menu system for the game is called “The Pod.” The music for the Pod is arranged in vertical layers that are activated and deactivated according to where the player is in the menu hierarchy. All the layers can be played simultaneously, and they play in multiple combinations… however, each of the individual layers is also associated with a specific portion of the menu system, and is activated when the player enters that particular part of the menu.
Let’s take a quick tour through the layers of the Pod menu music. I’ve embedded some short musical excerpts of each layer. You’ll find the SoundCloud players for each layer embedded below – just click the Play buttons to listen to each excerpt. The first layer of the Pod menu music is associated with the Main Menu, and it features some floaty, science-fiction-inspired textures and effects:
The next layer is associated with a menu labeled “My Levels,” and the music for that layer is very different. Now, woodwinds are accompanied by a gentle harp, combining to create a homey and down-to-earth mood:
Moving on to the music layer for the “Play” menu, we find that the instrumentation now features an ethereal choir and shimmering bells, expressing a much more celestial atmosphere:
Now let’s listen to the “Adventure” menu layer, in which plucked strings and bells combine to deliver a prominent melody line:
Finally, in the music layer associated with the “Community” and “Popit” menus, we hear a quirky mix of synths and effects that hearken back to menu music from previous games in the LittleBigPlanet franchise:
As the player navigates the Pod menu system, these various music layers are activated to correspond with the player’s location within the menu hierarchy. This sort of dynamic music triggering lies at the very heart of the Vertical Layering interactive music mechanism.
Every layer in a Vertical Layering composition can have a very distinct musical identity. When that layer is turned off, the entire mix changes in a noticeable way. The mix can be changed subtly…
… or it can be altered radically, with large scale activations or deactivations of layers. Even with these kinds of dramatic changes, the musical composition retains its identity. The same piece of music continues to play, and the player is conscious of continuing to hear the same musical composition, even though it has just altered in reaction to the circumstances of gameplay and the player’s progress.
In the Pod menu music system, the layers would change in reaction to the player’s menu navigation, which could be either slow and leisurely or brisk and purposeful. Layer activations and deactivations would occur with smooth crossfade transitions as the player moved from one menu to another. Now let’s take a look at a video showing some navigation through the Pod menu system, so we can hear how these musical layers behaved during actual gameplay:
As you can see, triggering unique musical layers for different portions of the menu system helps serve to define them. I hope you found this explanation of the Pod music to be interesting! If you attended GDC but missed my talk on the interactive music of LittleBigPlanet, you’ll be able to find the entire presentation posted as a video in the GDC Vault in just a few weeks. In the meantime, please feel free to add any comments or questions below!
The use of third-party audio middleware in game development is a slow-growth trend that will doubtless become more influential in the future, so I thought I’d devote my next two blog entries to some recent video tutorials produced by a few intrepid game audio pros who have stepped forward to help the community.
This first blog is devoted to Wwise, and the tutorials come to us courtesy of Michael Kamper, Senior Audio Developer at Telltale Games. With over 16 years of experience in audio production, Michael has served as Audio Director for The Bureau: Xcom Declassified, Bioshock 2 DLC, and Bioshock 2, among others. Michael has also enjoyed a successful career as a feature film sound designer for such movies as Mission Impossible III, The Day After Tomorrow, Legally Blonde, and many more. His experience in television includes sound design for Xena: Warrior Princess, Hercules: The Legendary Journeys and Profiler.
In the following two-part video tutorial, Michael generously details his Wwise workflow during music implementation for The Bureau: Xcom Declassified:
Wwise Interactive Music Demo – The Bureau – Part 1 – Switches
Wwise Interactive Music Demo – The Bureau – Part 2 – Segments/RTPCs