Tutti: Expressive Multi-Singer Synthesis via Structure-Level Timbre Control and Vocal Texture Modeling

TL;DR

Tutti is a novel framework for multi-singer singing voice synthesis that enables dynamic singer scheduling and detailed vocal texture modeling, improving realism and flexibility in complex multi-singer arrangements.

Contribution

It introduces a Structure-Aware Singer Prompt and a Condition-Guided VAE for capturing implicit acoustic textures, advancing multi-singer synthesis capabilities.

Findings

Outperforms existing systems in multi-singer scheduling accuracy.

Enhances acoustic realism in choral singing synthesis.

Demonstrates effective modeling of implicit vocal textures.

Abstract

While existing Singing Voice Synthesis systems achieve high-fidelity solo performances, they are constrained by global timbre control, failing to address dynamic multi-singer arrangement and vocal texture within a single song. To address this, we propose Tutti, a unified framework designed for structured multi-singer generation. Specifically, we introduce a Structure-Aware Singer Prompt to enable flexible singer scheduling evolving with musical structure, and propose Complementary Texture Learning via Condition-Guided VAE to capture implicit acoustic textures (e.g., spatial reverberation and spectral fusion) that are complementary to explicit controls. Experiments demonstrate that Tutti excels in precise multi-singer scheduling and significantly enhances the acoustic realism of choral generation, offering a novel paradigm for complex multi-singer arrangement. Audio samples are available at https://annoauth123-ctrl.github.io/Tutii_Demo/.