Music Generation using Human-In-The-Loop Reinforcement Learning

TL;DR

This paper introduces a real-time music generation system that uses human feedback within a reinforcement learning framework, guided by music theory principles, to iteratively improve compositions.

Contribution

It develops a novel Human-In-The-Loop Reinforcement Learning framework tailored for music, integrating music theory constraints with an episodic Q-learning algorithm.

Findings

System generates musical compositions with iterative human feedback.

Framework effectively incorporates music theory into reinforcement learning.

Produces compositions aligned with user preferences.

Abstract

This paper presents an approach that combines Human-In-The-Loop Reinforcement Learning (HITL RL) with principles derived from music theory to facilitate real-time generation of musical compositions. HITL RL, previously employed in diverse applications such as modelling humanoid robot mechanics and enhancing language models, harnesses human feedback to refine the training process. In this study, we develop a HILT RL framework that can leverage the constraints and principles in music theory. In particular, we propose an episodic tabular Q-learning algorithm with an epsilon-greedy exploration policy. The system generates musical tracks (compositions), continuously enhancing its quality through iterative human-in-the-loop feedback. The reward function for this process is the subjective musical taste of the user.