Dial up the Diegetics: Musical Sound Effects for the Video Game Composer (GDC 2024)

Hello everyone!  I’m video game composer Winifred Phillips, and I’m delighted to share that I’ll be delivering a lecture this March in San Francisco during the Game Developers Conference — one of the top industry conferences of the year.  My lecture is entitled “Dial Up the Diegetics: Musical Sound Effects,” and I was honored that GDC included it as a featured audio selection in the GDC 2024 Session Guide, and highlighted my participation as a GDC speaker this year by including me in the GDC 2024 Speaker Spotlight.  My lecture takes place on Thursday, March 21st in Room 3002 West Hall at the Moscone Center in San Francisco, and I’m really looking forward to this one!  We’ll be taking a detailed look at various ways that music and sound-design can cross fertilize each other.  Specifically, my presentation will focus on how environmental and incidental sound effects can be directed towards musical applications.  These audio assets, typically used by sound design experts, can serve to introduce quirky novelty into a game’s musical score.  They can also deepen player immersion in the environment of the game.

While I don’t want to spoil the content of my presentation, I thought I’d take this opportunity to break down an example of this technique from one of my projects.  This particular example was regrettably cut from my presentation due to time constraints, which affords me the opportunity to share it here instead!  While my lecture will have a much wider scope on the topic of musical diegetics, this article will drill down on one specific case-study.  So let’s start with some basic definitions.

The Oxford Reference does an awesome job of defining the core principle of Diegesis, albeit confining that definition to the world of film theory:

With this in mind, we can consider diegetics in video game design to be elements that fall firmly inside the “spatio-temporal world” that’s been established by the game.  For instance, if a character in a WWII first-person shooter finds a rifle on the ground and picks it up, that rifle is diegetic.  It exists in the world.  However, if the character starts trying to load that rifle and an options menu appears onscreen to allow the player to select appropriate ammo, that menu screen is “extradiegetic” (often called non-diegetic).  We accept that a floating menu screen like this one wouldn’t exist in the “spatio-temporal world” of a WWII narrative.

Moving this into the realm of game sound, if our WWII soldier loads his rifle and hears it click, that sound is diegetic.  If he then puts his remaining rifle rounds away in his pack and the game emits a little warning beep to indicate that ammo is low, that sound is non-diegetic.

Let’s focus more narrowly on our discipline as game composers.  The great majority of musical content in any game is non-diegetic.  As an example, perhaps a stirring orchestral soundtrack accompanies our WWII soldier as he moves carefully through an obliterated French village.  This music isn’t being performed by exhausted local musicians fiddling away from some crumbling rooftop — it exists outside of the “spatio-temporal world” of the game.  With a few exceptions, most of our work as game composers will consist of purely non-diegetic music composition.  However, that doesn’t mean that we are completely cut off from diegetic tools and techniques.

When we want to fold diegetics into our music, we can first ask ourselves, “what is my sound palette in this diegetic world?  If I were walking around in the environment of the game and I had to make music with materials that I might find there, what would they be?”  Considering this, we can begin gathering an assortment of appropriate noises and environmental sounds, and then experiment with ways they can be used rhythmically and tonally.  Going further, we can extrapolate from the nature of these aural diegetics to add related sounds that feel congruous with the environment, even if they aren’t directly visible in the landscape at hand.  Anything that feels consistent with the sonic texture of the game can become a source of inspiration, and a new musical instrument at our disposal.

That’s the whole philosophy behind diegetics in music.  Using this technique, we can open up a whole new array of sonic tools for ourselves, and in the process lend our music a distinct character that ties it more closely to the game’s “spatio-temporal world.”  In my GDC lecture, I’ll be exploring a wide array of examples of this technique as employed in my projects as a game composer, but now let’s look at a single example from one of my more whimsical projects — the popular LittleBigPlanet 2 Cross Controller game!

LittleBigPlanet is a blockbuster game franchise starring one of Playstation’s most famous and lovable mascots— Sackboy!  I composed the entire musical score for LittleBigPlanet 2 Cross Controller, including the music for the Hen-o-morph Encounter level.  In this level. our hero Sackboy must make his way through a rusted and grimy spaceship full of spinning gears, pipes, and red-hot vents spitting steam.  With all this in mind, and taking into account the personality of the game’s little knitted hero, it seemed incredibly obvious to me that such a gritty space adventure would need to be accompanied by disco music.  I laid down a funky groove with some oddball percussion and a string section.  Let’s check out a relatively brief excerpt from the full composition, to get an idea of how the mix sounded at this point:

Having this general disco groove in place, I turned my attention to the idea of aural diegetics evocative of the environment.  Time to start hunting for inspirational sound effects and exploring their musical possibilities!  I began with the forceful hiss of steam, along with sounds of a gas burner igniting:

With things starting to sound sizzling-hot, I turned my attention to all that metal and rust everywhere.  Clearly, metallics would go a long way towards immersing players in this environment, so I started gathering and experimenting with lots of metal impact sounds:

So now we’ve included sounds evocative of heat and metal.  Time to address the underlying machinery!  I searched for interesting machine sounds that could be used in a musical way, and one of the best sounds of this type was actually an old-school printing press.  Combining that with the sound of air pumps and an assembly-line robot produced some fun effects:

With these machine sounds in place, I now turned my attention to all of those spinning gears.  This was much trickier to pin down.  Failing to find any literal “gear” sounds that fit the bill, I stumbled across something that at first seemed like an unintuitive solution: an old wind-up toy.  Here’s what the initial recording of that toy sounded like:

I edited the wind-up toy recording into 28 snippets of sound, which I ran through some signal processing and then spread across my keyboard so that I could perform rhythmic patterns with it:

After that, I focused on an assortment of purely electronic sounds that would feel like diegetic elements originating from pieces of equipment in the environment.

Finally, I pulled together some beeps and alarms that were evocative of a sci-fi setting:

All right, let’s check out the full mix of that same excerpt we listened to earlier, to see how the diegetic effects influenced the impact of the overall composition:

Finally, here’s a gameplay video that shows what it was like playing the Hen-O-Morph Encounter level, with the diegetic-rich musical score helping to immerse players in the atmosphere of the level:

Over the years, I’ve had a fantastic time working diegetic aural elements into my musical compositions, and I’m looking forward to sharing lots more examples and strategies during my GDC speech, “Dial Up the Diegetics: Musical Sound Effects.”  I hope I’ll see you there!  In the meantime, you can read more about game music composition in my book, A Composer’s Guide to Game Music.  Thanks for reading!

Winifred Phillips is a BAFTA-nominated video game composer.  The music she composed for one of her most recent video game projects (Secrets of Skeifa Island) won two Global Music Award Gold Medals, two NYX Gold Awards, and was nominated for a Society of Composers and Lyricists Award.  Music from Secrets of Skeifa Island is included in her latest album release, 

Ancient Heroes, released by the BMG record label 1 Revolution.  Other recent game projects include Wizardry: Proving Grounds of the Mad Overlord, and the hit PlayStation 5 launch title Sackboy: A Big Adventure (soundtrack album now available).  Popular music from Phillips’ award-winning Assassin’s Creed Liberation score was featured in the performance repertoire of the Assassin’s Creed Symphony World Tour, which made its Paris debut with an 80-piece orchestra and choir. As an accomplished video game composer, Phillips is best known for composing music for games in many of the most famous and popular franchises in gaming: the list includes Assassin’s Creed, God of War, Total War, 

The Sims, Sackboy / LittleBigPlanet, Lineage, and Wizardry.  Phillips’ has received numerous awards, including an Interactive Achievement Award / D.I.C.E. Award from the Academy of Interactive Arts and Sciences, six Game Audio Network Guild Awards (including Music of the Year), and four Hollywood Music in Media Awards. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. As one of the foremost authorities on music for interactive entertainment, Winifred Phillips has given lectures at the Library of Congress in Washington DC, the Society of Composers and Lyricists, the Game Developers Conference, the Audio Engineering Society, and many more. Phillips’ enthusiastic fans showered her with questions during a Reddit Ask-Me-Anything session that went viral, hit the Reddit front page, received 14.9 thousand upvotes, and became one of the most popular gaming AMAs ever hosted on Reddit. An interview with her has been published as a part of the Routledge text, Women’s Music for the Screen: Diverse Narratives in Sound, which collects the viewpoints of the most esteemed female composers in film, television, and games. Follow her on Twitter, Facebook, Instagram, and Threads.