ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Models

TL;DR

ViTex introduces a novel visual texture control method for multi-track symbolic music generation using discrete diffusion models, enabling explicit control over instrument textures and roles in compositions.

Contribution

The paper presents ViTex, a new visual representation for instrumental textures and a diffusion model conditioned on this representation for controllable multi-track music generation.

Findings

Effective control over instrument textures and roles.

High-quality unconditional music generation.

Accessible demo and code available online.

Abstract

In automatic music generation, a central challenge is to design controls that enable meaningful human-machine interaction. Existing systems often rely on extrinsic inputs such as text prompts or metadata, which do not allow humans to directly shape the composition. While prior work has explored intrinsic controls such as chords or hierarchical structure, these approaches mainly address piano or vocal-accompaniment settings, leaving multitrack symbolic music largely underexplored. We identify instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and propose ViTex, a visual representation of instrumental texture. In ViTex, color encodes instrument choice, spatial position represents pitch and time, and stroke properties capture local textures. Building on this representation, we develop a discrete diffusion model conditioned on ViTex and chord progressions to generate 8-measure multi-track symbolic music, enabling explicit texture-level control while maintaining strong unconditional generation quality. The demo page and code are avaliable at https://vitex2025.github.io/.