Audio style transfer
Eric Grinstein, Ngoc Duong, Alexey Ozerov, Patrick P\'erez

TL;DR
This paper introduces a novel audio style transfer framework that adapts texture synthesis techniques, using a sound texture model and optimization, to transfer style from reference audio to target content, differing from visual methods.
Contribution
The paper presents a flexible audio style transfer method that initializes from target content and focuses on texture, not structure, with experiments on various audio signals.
Findings
Effective transfer of style in audio signals
Initialization from target content improves results
Potential demonstrated across different audio types
Abstract
'Style transfer' among images has recently emerged as a very active research topic, fuelled by the power of convolution neural networks (CNNs), and has become fast a very popular technology in social media. This paper investigates the analogous problem in the audio domain: How to transfer the style of a reference audio signal to a target audio content? We propose a flexible framework for the task, which uses a sound texture model to extract statistics characterizing the reference audio style, followed by an optimization-based audio texture synthesis to modify the target content. In contrast to mainstream optimization-based visual transfer method, the proposed process is initialized by the target content instead of random noise and the optimized loss is only about texture, not structure. These differences proved key for audio style transfer in our experiments. In order to extract…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpeech and Audio Processing · Music and Audio Processing · Speech Recognition and Synthesis
