Sounderfeit: Cloning a Physical Model with Conditional Adversarial Autoencoders

TL;DR

This paper introduces Sounderfeit, a neural network-based system that clones physical model sounds by learning parameter-signal relationships, enabling real-time, data-driven sound synthesis and parameter estimation.

Contribution

It presents a novel adversarial autoencoder approach conditioned on physical parameters for accurate sound cloning and resynthesis, with real-time synthesis capabilities.

Findings

Adversarial training improves parameter estimation accuracy.

The system can effectively clone and resynthesize physical model sounds.

Real-time synthesis is feasible with the proposed neural network architecture.

Abstract

An adversarial autoencoder conditioned on known parameters of a physical modeling bowed string synthesizer is evaluated for use in parameter estimation and resynthesis tasks. Latent dimensions are provided to capture variance not explained by the conditional parameters. Results are compared with and without the adversarial training, and a system capable of "copying" a given parameter-signal bidirectional relationship is examined. A real-time synthesis system built on a generative, conditioned and regularized neural network is presented, allowing to construct engaging sound synthesizers based purely on recorded data.