Spatial Voice Conversion: Voice Conversion Preserving Spatial Information and Non-target Signals

TL;DR

This paper introduces spatial voice conversion, a novel task that converts target voices while maintaining spatial cues and non-target signals, addressing the limitations of traditional single-channel methods.

Contribution

It presents a baseline approach combining BSS, VC, and spatial mixing for multi-channel waveforms, and establishes a benchmark for future research in spatial voice conversion.

Findings

Identified key challenges in preserving spatial information and audio quality.

Demonstrated the fundamental difficulties in balancing spatial cues and voice conversion.

Provided a publicly available codebase to facilitate further research.

Abstract

This paper proposes a new task called spatial voice conversion, which aims to convert a target voice while preserving spatial information and non-target signals. Traditional voice conversion methods focus on single-channel waveforms, ignoring the stereo listening experience inherent in human hearing. Our baseline approach addresses this gap by integrating blind source separation (BSS), voice conversion (VC), and spatial mixing to handle multi-channel waveforms. Through experimental evaluations, we organize and identify the key challenges inherent in this task, such as maintaining audio quality and accurately preserving spatial information. Our results highlight the fundamental difficulties in balancing these aspects, providing a benchmark for future research in spatial voice conversion. The proposed method's code is publicly available to encourage further exploration in this domain.