# Continuous descriptor-based control for deep audio synthesis

## TL;DR

This paper introduces a lightweight deep audio generative model with continuous, descriptor-based control, enabling expressive real-time sound synthesis suitable for integration into musical workflows and hardware devices.

## Contribution

A novel deep generative audio model that offers explicit, continuous control over sound features, combining adversarial training with conditioning for versatile sound synthesis.

## Key findings

- Effective control over diverse sound types
- Real-time performance with embedded hardware compatibility
- Versatile timbre and attribute transfer capabilities

## Abstract

Despite significant advances in deep models for music generation, the use of these techniques remains restricted to expert users. Before being democratized among musicians, generative models must first provide expressive control over the generation, as this conditions the integration of deep generative models in creative workflows. In this paper, we tackle this issue by introducing a deep generative audio model providing expressive and continuous descriptor-based control, while remaining lightweight enough to be embedded in a hardware synthesizer. We enforce the controllability of real-time generation by explicitly removing salient musical features in the latent space using an adversarial confusion criterion. User-specified features are then reintroduced as additional conditioning information, allowing for continuous control of the generation, akin to a synthesizer knob. We assess the performance of our method on a wide variety of sounds including instrumental, percussive and speech recordings while providing both timbre and attributes transfer, allowing new ways of generating sounds.