DNN-based ensemble singing voice synthesis with interactions between singers

TL;DR

This paper introduces a novel DNN-based singing voice synthesis method that models interactions between singers to produce more unified ensemble singing voices, addressing limitations of solo-focused synthesis methods.

Contribution

It proposes a new SVS architecture that incorporates interactions between singers using musical scores and specialized loss functions, enhancing ensemble unity.

Findings

Improved vocal ensemble unity in synthesized singing.

Effective modeling of singer interactions in DNN-based SVS.

Enhanced realism and cohesion in ensemble singing synthesis.

Abstract

We propose a singing voice synthesis (SVS) method for a more unified ensemble singing voice by modeling interactions between singers. Most existing SVS methods aim to synthesize a solo voice, and do not consider interactions between singers, i.e., adjusting one's own voice to the others' voices. Since the production of ensemble voices from solo singing voices ignores the interactions, it can degrade the unity of the vocal ensemble. Therefore, we propose a SVS that reproduces the interactions. It is based on an architecture that uses musical scores of multiple voice parts, and loss functions that simulate the interactions' effect to acoustic features. Experimental results show that our methods improve the unity of the vocal ensemble.