DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matching

TL;DR

DAFMSVC is a novel singing voice conversion method that uses dual attention and flow matching to improve timbre similarity and audio quality, effectively handling unseen speaker voices without degradation.

Contribution

The paper introduces DAFMSVC, combining a dual attention mechanism and flow matching with SSL features to address timbre leakage and quality issues in one-shot singing voice conversion.

Findings

Significantly improves timbre similarity and naturalness.

Outperforms state-of-the-art methods in evaluations.

Effectively prevents timbre leakage in unseen speakers.

Abstract

Singing Voice Conversion (SVC) transfers a source singer's timbre to a target while keeping melody and lyrics. The key challenge in any-to-any SVC is adapting unseen speaker timbres to source audio without quality degradation. Existing methods either face timbre leakage or fail to achieve satisfactory timbre similarity and quality in the generated audio. To address these challenges, we propose DAFMSVC, where the self-supervised learning (SSL) features from the source audio are replaced with the most similar SSL features from the target audio to prevent timbre leakage. It also incorporates a dual cross-attention mechanism for the adaptive fusion of speaker embeddings, melody, and linguistic content. Additionally, we introduce a flow matching module for high quality audio generation from the fused features. Experimental results show that DAFMSVC significantly enhances timbre similarity and naturalness, outperforming state-of-the-art methods in both subjective and objective evaluations.