DiffuseRoll: Multi-track multi-category music generation based on diffusion model

TL;DR

DiffuseRoll introduces a diffusion-based approach for generating complex multi-track, multi-attribute symphonic music by encoding music attributes into color-coded images, outperforming existing methods in polyphonic music generation.

Contribution

The paper presents a novel diffusion model that encodes multi-attribute music into color-coded images for improved multi-track, multi-attribute music generation.

Findings

Outperforms state-of-the-art in polyphonic music generation

Effectively encodes rich attributes using color coding scheme

Produces high-quality multi-track symphonic music

Abstract

Recent advancements in generative models have shown remarkable progress in music generation. However, most existing methods focus on generating monophonic or homophonic music, while the generation of polyphonic and multi-track music with rich attributes is still a challenging task. In this paper, we propose a novel approach for multi-track, multi-attribute symphonic music generation using the diffusion model. Specifically, we generate piano-roll representations with a diffusion model and map them to MIDI format for output. To capture rich attribute information, we introduce a color coding scheme to encode note sequences into color and position information that represents pitch,velocity, and instrument. This scheme enables a seamless mapping between discrete music sequences and continuous images. We also propose a post-processing method to optimize the generated scores for better performance. Experimental results show that our method outperforms state-of-the-art methods in terms of polyphonic music generation with rich attribute information compared to the figure methods.