TEAdapter: Supply abundant guidance for controllable text-to-music generation

TL;DR

TEAdapter is a lightweight plugin that enhances controllability in text-to-music generation, allowing users to specify diverse control conditions for more intricate and structured music creation.

Contribution

We introduce TEAdapter, a novel plugin that provides fine-grained control over music generation, adaptable to various diffusion models and capable of managing complex structural controls.

Findings

Enables precise control over music generation at multiple levels.

Ensures high-quality output with diverse control conditions.

Transferable to different diffusion model architectures.

Abstract

Although current text-guided music generation technology can cope with simple creative scenarios, achieving fine-grained control over individual text-modality conditions remains challenging as user demands become more intricate. Accordingly, we introduce the TEAcher Adapter (TEAdapter), a compact plugin designed to guide the generation process with diverse control information provided by users. In addition, we explore the controllable generation of extended music by leveraging TEAdapter control groups trained on data of distinct structural functionalities. In general, we consider controls over global, elemental, and structural levels. Experimental results demonstrate that the proposed TEAdapter enables multiple precise controls and ensures high-quality music generation. Our module is also lightweight and transferable to any diffusion model architecture. Available code and demos will be found soon at https://github.com/Ashley1101/TEAdapter.