Sounderfeit: Cloning a Physical Model using a Conditional Adversarial Autoencoder

TL;DR

This paper introduces Sounderfeit, a neural network-based system that clones physical model sounds by learning parameter-signal relationships, enabling real-time, data-driven sound synthesis and parameter estimation.

Contribution

It presents a novel adversarial autoencoder approach conditioned on physical model parameters for accurate sound cloning and real-time synthesis from recorded data.

Findings

Adversarial training improves parameter estimation accuracy.

The system can effectively clone and resynthesize physical model sounds.

Real-time synthesis is feasible with the proposed neural network architecture.

Abstract

An adversarial autoencoder conditioned on known parameters of a physical modeling bowed string synthesizer is evaluated for use in parameter estimation and resynthesis tasks. Latent dimensions are provided to capture variance not explained by the conditional parameters. Results are compared with and without the adversarial training, and a system capable of "copying" a given parameter-signal bidirectional relationship is examined. A real-time synthesis system built on a generative, conditioned and regularized neural network is presented, allowing to construct engaging sound synthesizers based purely on recorded data.