Conditioning Autoencoder Latent Spaces for Real-Time Timbre Interpolation and Synthesis

TL;DR

This paper compares autoencoder topologies for timbre generation, showing how activation functions affect embedding distribution, and introduces chroma conditioning for improved real-time timbre synthesis.

Contribution

It demonstrates the impact of activation functions on embedding distribution and proposes chroma feature conditioning for enhanced timbre interpolation and synthesis.

Findings

Sigmoid activation yields more bounded embeddings.

Chroma conditioning improves timbre representation.

Open source real-time synthesis algorithm provided.

Abstract

We compare standard autoencoder topologies' performances for timbre generation. We demonstrate how different activation functions used in the autoencoder's bottleneck distributes a training corpus's embedding. We show that the choice of sigmoid activation in the bottleneck produces a more bounded and uniformly distributed embedding than a leaky rectified linear unit activation. We propose a one-hot encoded chroma feature vector for use in both input augmentation and latent space conditioning. We measure the performance of these networks, and characterize the latent embeddings that arise from the use of this chroma conditioning vector. An open source, real-time timbre synthesis algorithm in Python is outlined and shared.