MIDI-Informed Singing Accompaniment Generation in a Compositional Song Pipeline

TL;DR

This paper introduces MIDI-SAG, a structured, long-form singing accompaniment generation system that uses MIDI and chord information to improve coherence and control in professional music composition.

Contribution

MIDI-SAG is a novel framework that incorporates symbolic timing and structure planning for stable, long-form, score-to-song generation, trained efficiently with pre-trained modules.

Findings

MIDI-SAG can generate coherent long-form singing accompaniments.

The system effectively integrates MIDI and chord data for structured music synthesis.

Pre-trained modules enable data-efficient training on a single GPU.

Abstract

While end-to-end lyrics-to-song models offer convenience for casual users, professional songwriters require score-to-song systems that allow them to retain authorship over the core melody. However, existing score-to-song methods are limited to short-form snippets and fail to maintain coherence in long-form generation, particularly during vocal-silent sections like intros and bridges. To address this long-form bottleneck, we propose MIDI-informed singing accompaniment generation (MIDI-SAG). Unlike conventional audio-only models, MIDI-SAG utilizes symbolic timing and chord information derived from the vocal MIDI to provide a stable musical roadmap. By incorporating structure planning, which defines temporal boundaries and semantic labels, our framework facilitates consistent generation across both vocal and non-vocal sections. We demonstrate the feasibility of this compositional pipeline by leveraging specialized pre-trained modules, enabling data-efficient training on a single GPU. Our experiments show the potential of this approach for both professional score-to-song and general lyrics-to-song tasks. While an early exploration, MIDI-SAG suggests a promising direction for structured, long-form music synthesis. Audio demos are available, and the code will be open-sourced at https://composerflow.github.io/web_revealed/.