Malakai: Music That Adapts to the Shape of Emotions

TL;DR

Malakai is a tool that uses machine learning and procedural algorithms to generate real-time, emotion-adaptive music for interactive experiences, enabling users to create, remix, and share dynamic songs.

Contribution

It introduces Malakai, a novel platform combining ML models and procedural methods to produce real-time, emotion-responsive music for diverse user interaction.

Findings

Enables real-time adaptation of music to emotional states

Supports user interaction and remixing of dynamic songs

Facilitates creation and sharing of emotion-aware musical compositions

Abstract

The advent of ML music models such as Google Magenta's MusicVAE now allow us to extract and replicate compositional features from otherwise complex datasets. These models allow computational composers to parameterize abstract variables such as style and mood. By leveraging these models and combining them with procedural algorithms from the last few decades, it is possible to create a dynamic song that composes music in real-time to accompany interactive experiences. Malakai is a tool that helps users of varying skill levels create, listen to, remix and share such dynamic songs. Using Malakai, a Composer can create a dynamic song that can be interacted with by a Listener