Network Modulation Synthesis: New Algorithms for Generating Musical Audio Using Autoencoder Networks

TL;DR

This paper introduces network modulation synthesis, a novel framework utilizing autoencoder networks to facilitate the creation of complex, time-varying musical audio, making it easier to explore the latent space for innovative sound design.

Contribution

The paper presents new algorithms for autoencoder-based audio synthesis, enabling intuitive navigation of the latent space and expanding creative possibilities in musical audio generation.

Findings

Algorithms enable easy generation of time-varying parameters

Spectrogram analysis confirms effective control over audio features

Open-source implementation available for broader use

Abstract

A new framework is presented for generating musical audio using autoencoder neural networks. With the presented framework, called network modulation synthesis, users can create synthesis architectures and use novel generative algorithms to more easily move through the complex latent parameter space of an autoencoder model to create audio. Implementations of the new algorithms are provided for the open-source CANNe synthesizer network, and can be applied to other autoencoder networks for audio synthesis. Spectrograms and time-series encoding analysis demonstrate that the new algorithms provide simple mechanisms for users to generate time-varying parameter combinations, and therefore auditory possibilities, that are difficult to create by generating audio from handcrafted encodings.