Tag Archives: Music

How AI Revolutionizes Music Streaming

In 2020, worldwide music streaming revenue hit 11.4 billion dollars, a 2800% growth over the course of a decade. Three hundred forty-one million paid online streaming subscribers get their music from top services like Apple Music, Spotify, and Tidal. The competition for listeners is fierce. Each company looks to leverage every advantage they can in pursuit of higher market share.

Like all major tech conglomerates, music streaming services collect an exceptional amount of user data through their platforms and are creating elaborate AI algorithms designed to improve user experience on a number of levels. Spotify has emerged as the largest on-demand music service active today and bolstered its success through the innovative use of AI.

Here are the top ways in which AI has changed music streaming:

COLLABORATIVE FILTERING

AI has the ability to sift through a plenitude of implicit consumer data, including:

  • Song preferences
  • Keyword preferences
  • Playlist data
  • Geographic location of listeners
  • Most used devices

AI algorithms can analyze user trends and identify users with similar tastes. For example, if AI deduces that User 1 and User 2 have similar tastes, then it can infer that songs User 1 has liked will also be enjoyed by User 2. Spotify’s algorithms will leverage this information to provide recommendations for User 2 based on what User 1 likes, but User 2 has yet to hear.

The result is not only improved recommendations, but greater exposure for artists that otherwise may not have been organically found by User 2.

NATURAL LANGUAGE PROCESSING

Natural Language Processing is a burgeoning field in AI. Previously in our blog, we covered GPT-3, the latest Natural Language Processing (NLP) technology developed by OpenAI. Music streaming services are well-versed in the technology and leverage it in a variety of ways to enhance UI.

nlp

Algorithms scan a track’s metadata, in addition to blog posts, discussions, and news articles about artists or songs on the internet to determine connections. When artists/songs are mentioned alongside artists/songs the user likes, algorithms make connections that fuel future recommendations.

GPT-3 is not perfect; its ability to track sentiments lacks nuance. As Sonos Radio general manager Ryan Taylor recently said to Fortune Magazine: “The truth is music is entirely subjective… There’s a reason why you listen to Anderson .Paak instead of a song that sounds exactly like Anderson .Paak.”

As NLP technology evolves and algorithms extend their grasp of the nuances of language, so will the recommendations provided to you by music streaming services.

AUDIO MODELS

1_5hAP-k77FKJVG1m1qqdpYA

AI can study audio models to categorize songs exclusively based on their waveforms. This scientific, binary approach to analyzing creative work enables streaming services to categorize songs and create recommendations regardless of the amount of coverage a song or artist has received.

BLOCKCHAIN

Artist payment of royalties on streaming services poses its own challenges, problems, and short-comings. Royalties are deduced from trillions of data points. Luckily, blockchain is helping to facilitate a smoother artist’s payment process. Blockchain technology can not only make the process more transparent but also more efficient. Spotify recently acquired blockchain company Mediachain Labs, which will, many pundits are saying, change royalty payments in streaming forever.

MORE TO COME

While AI has vastly improved streaming ability to keep their subscribers compelled, a long road of evolution lies ahead before it can come to a deep understanding of what motivates our musical tastes and interests. Today’s NLP capabilities provided by GPT-3 will probably become fairly archaic within three years as the technology is pushed further. One thing is clear: as streaming companies amass decades’ worth of user data, they won’t hesitate to leverage it in their pursuit of market dominance.

The Masterful Audio of The Legend of Zelda: Breath of the Wild

The original Legend of Zelda came out in 1986 on the NES. Games were evolving from endurance tests to simple narratives in the Socratic tradition. Having recently stunned the world with Super Mario Bros in 1985, legendary game designer Shigeru Miyamoto set-out to create a game with a world. Miyamoto grew up in Japan exploring the fields, woods, and caves of Kyoto. He designed the first Legend of Zelda to be “a miniature garden that players can put inside their drawer.” He combined Peter Pan, Tinker Bell, and the Arthurian legend of the sword in the stone to create what would become one of Nintendo’s greatest franchises.

Zelda Creator Shigeru Miyamoto

Zelda Creator Shigeru Miyamoto

31 years and over 18 iterations later, Nintendo released The Legend of Zelda: Breath of the Wild for Nintendo Switch to widespread critical acclaim—many publications have declared BOTW to be among the greatest games ever created.

As any game designer knows, audio is vital to providing an audience with feedback. As with every element of the game, Nintendo has taken Zelda’s audio to a new level with Breath of the Wild. Here is our review and breakdown of what makes the audio of BOTW so special:

MUSIC

Nintendo composer Koji Kondo initially intended to use Maurice Ravel’s Bolero as the theme music for the original Legend of Zelda in 1985, but upon learning the copyright was intact, he composed the legendary “Zelda Theme”. Nintendo has always dedicated a large amount of time and care to the music of Zelda, but with BOTW, they’ve taken things to the next level.

Acute Zelda-philes will recognize the integration of electronic and orchestral instrumentation as innovative to the franchise. Music has always been used to give the user feedback regarding the setting— Dungeon Theme, which plays when Link is in a dungeon level.

In music, this technique is called a leitmotif: a recurrent theme throughout a musical or literary composition, associated with a particular person, idea, or situation.

In BOTW, music is used to delineate location, themes, characters, and even special enemies! While the combat theme plays fighting most baddies, anyone who has played BOTW will recognize the infamous Guardian Theme and of course the Hinox Theme.

The first time the player hears these songs come on, it is scary and immediately invests you with a bombastic sense of imminent danger.

Music changes by location and by time of day, as demonstrated by the Gerudo Town Day and Night songs.

In the Korok Forest, when entering uncharted territory, the user hears an eerie tune that gradually increases in volume until the user is transported back to the start of the forest, as demonstrated in this video:

https://youtu.be/7FgZw1geaHw

Shards of stories are told through musical sequences sung by Kass, dispersing exposition about the world and clues to the puzzles the user must solve.

All told, The Legend of Zelda Soundtrack has about 211 songs! Dedicated players can easily invest hundreds of hours into an open-world RPG, so diversifying the music is vital to keeping players engaged and preventing  the game from feeling repetitive. Big, sweeping orchestral soundtracks like these are saving symphony orchestras, according to the Wall Street Journal.

SOUND DESIGN

Longtime composer and sound supervisor Hajime Wakai oversaw the sound of BOTW. While audio cues for sound effects in the Zelda series traditionally have had a very lo-fi electronic feel, BOTW uses fully orchestrated, live-performed instruments, investing the user with the feeling of watching a live performance. Wakai recorded over 10,000 different sounds for the game. In the Nintendo blog, he explains the many different types of footsteps.

“I mixed various kinds of sand so to get good footsteps. We searched for a combination of materials and how to apply them for the sound of the equipment. Ice boards melt and crack each time they walk and have to be replaced. The studio is full of stinky smells as you continue stepping on the grass. There were various hardships.”

Check out these photos from inside the sound studio:

production-notes10-image01

production-notes10-image02

The intricacies of the sound design are on full display in BOTW, with different sounds for hitting flesh creatures, hitting bone creatures and the very notable sound of the Master Sword clashing against a Guardian:

https://www.youtube.com/watch?v=mXkmknLZiFE&feature=youtu.be

The sound of BOTW enhance the experience and helps the user FEEL the game. Perhaps no better example of Sound Design doing this is the sound effect made when a user breaks their weapon:

https://www.youtube.com/watch?v=g9QLzmH4CFI&feature=youtu.be

Perhaps the biggest departure from the Zelda series is the use of voices. While Zelda games traditionally only represents dialogue with text and strange grunts, BOTW includes cut-scenes with fully-voiced Zelda, Daruth the Goron, Mipha the Zora, and more. BOTW incentivizes accruing these cut scenes or “Shattered Memories” by having them be collectable. Only after collecting all of the “Shattered Memories” can the user watch the story of the game in full from front to back.

OVERALL

Given the grand scale of BOTW, it’s no surprise that the game overdelivers when it comes to its soundscape. Nintendo’s first major entry into the Zelda series in 6 years is a masterpiece. Game designers take note of what is perhaps the best sound design ever created for a video game.