Sing-On-Your-Beat: Simple Text-Controllable Accompaniment Generations

TL;DR

This paper introduces a simple text-controlled method for generating musical accompaniments that align with vocals, instrumentation, and genre, enabling more precise and customizable music creation.

Contribution

It presents a novel, straightforward approach for text-controllable accompaniment generation that improves alignment with specified musical attributes.

Findings

Successfully generated 10-second accompaniments with vocal input and text prompts

Demonstrated effective control over instrumentation and genre alignment

Achieved high-quality accompaniment generation through extensive experiments

Abstract

Singing is one of the most cherished forms of human entertainment. However, creating a beautiful song requires an accompaniment that complements the vocals and aligns well with the song instruments and genre. With advancements in deep learning, previous research has focused on generating suitable accompaniments but often lacks precise alignment with the desired instrumentation and genre. To address this, we propose a straightforward method that enables control over the accompaniment through text prompts, allowing the generation of music that complements the vocals and aligns with the song instrumental and genre requirements. Through extensive experiments, we successfully generate 10-second accompaniments using vocal input and text control.