SoundMorpher: Perceptually-Uniform Sound Morphing with Diffusion Model

TL;DR

SoundMorpher introduces a perceptually-uniform sound morphing method using diffusion models and perceptual metrics, enabling high-quality sound transitions for creative and industrial applications.

Contribution

It proposes a novel sound morphing approach that explicitly models perceptual differences and introduces new evaluation metrics for sound morphing quality.

Findings

Effective in generating perceptually uniform sound transitions

Outperforms traditional linear interpolation methods

Versatile for real-world audio applications

Abstract

We present SoundMorpher, an open-world sound morphing method designed to generate perceptually uniform morphing trajectories. Traditional sound morphing techniques typically assume a linear relationship between the morphing factor and sound perception, achieving smooth transitions by linearly interpolating the semantic features of source and target sounds while gradually adjusting the morphing factor. However, these methods oversimplify the complexities of sound perception, resulting in limitations in morphing quality. In contrast, SoundMorpher explores an explicit relationship between the morphing factor and the perception of morphed sounds, leveraging log Mel-spectrogram features. This approach further refines the morphing sequence by ensuring a constant target perceptual difference for each transition and determining the corresponding morphing factors using binary search. To address the lack of a formal quantitative evaluation framework for sound morphing, we propose a set of metrics based on three established objective criteria. These metrics enable comprehensive assessment of morphed results and facilitate direct comparisons between methods, fostering advancements in sound morphing research. Extensive experiments demonstrate the effectiveness and versatility of SoundMorpher in real-world scenarios, showcasing its potential in applications such as creative music composition, film post-production, and interactive audio technologies. Our demonstration and codes are available at~\url{https://xinleiniu.github.io/SoundMorpher-demo/}.