Prompt-guided Precise Audio Editing with Diffusion Models

TL;DR

This paper introduces PPAE, a training-free, prompt-guided method for precise audio editing using diffusion models, leveraging cross-attention maps for accurate local modifications and a hierarchical pipeline for smooth results.

Contribution

It proposes a novel, training-free audio editing framework that utilizes cross-attention maps and a hierarchical pipeline to enable precise, prompt-guided modifications within audio tracks.

Findings

Effective local editing demonstrated in various tasks

Hierarchical pipeline improves editing smoothness

Utilizes cross-attention maps for accuracy

Abstract

Audio editing involves the arbitrary manipulation of audio content through precise control. Although text-guided diffusion models have made significant advancements in text-to-audio generation, they still face challenges in finding a flexible and precise way to modify target events within an audio track. We present a novel approach, referred to as PPAE, which serves as a general module for diffusion models and enables precise audio editing. The editing is based on the input textual prompt only and is entirely training-free. We exploit the cross-attention maps of diffusion models to facilitate accurate local editing and employ a hierarchical local-global pipeline to ensure a smoother editing process. Experimental results highlight the effectiveness of our method in various editing tasks.