Controllable and Interpretable Singing Voice Decomposition via Assem-VC

TL;DR

This paper introduces Assem-VC, a system for controllable and interpretable singing voice decomposition and synthesis, enabling personalized singing voice conversion with perfect synchronization.

Contribution

The paper presents a novel singing voice decomposition method that encodes linguistic content, pitch, and speaker identity for high-quality voice conversion.

Findings

Achieved perfectly synchronized duet singing voices.

Successfully converted singing voices with target speaker embedding.

Demonstrated controllable and interpretable voice synthesis.

Abstract

We propose a singing decomposition system that encodes time-aligned linguistic content, pitch, and source speaker identity via Assem-VC. With decomposed speaker-independent information and the target speaker's embedding, we could synthesize the singing voice of the target speaker. In conclusion, we made a perfectly synced duet with the user's singing voice and the target singer's converted singing voice.