Modeling Animal Vocalizations through Synthesizers

TL;DR

This paper explores using differentiable modular synthesizers to model, emulate, and generate animal vocalizations, offering a controllable and interpretable alternative to neural network-based sound models.

Contribution

It introduces an optimization framework for inferring synthesizer parameters for animal sounds, enhancing control and interpretability over sound modeling.

Findings

Gradient-based and genetic algorithms effectively infer synthesizer parameters.

Synthesizer parameters enable controllable and interpretable animal sound modeling.

The approach offers a flexible alternative to neural network models for bioacoustic synthesis.

Abstract

Modeling real-world sound is a fundamental problem in the creative use of machine learning and many other fields, including human speech processing and bioacoustics. Transformer-based generative models and some prior work (e.g., DDSP) are known to produce realistic sound, although they have limited control and are hard to interpret. As an alternative, we aim to use modular synthesizers, i.e., compositional, parametric electronic musical instruments, for modeling non-music sounds. However, inferring synthesizer parameters given a target sound, i.e., the parameter inference task, is not trivial for general sounds, and past research has typically focused on musical sound. In this work, we optimize a differentiable synthesizer from TorchSynth in order to model, emulate, and creatively generate animal vocalizations. We compare an array of optimization methods, from gradient-based search to genetic algorithms, for inferring its parameters, and then demonstrate how one can control and interpret the parameters for modeling non-music sounds.