Example-Based Framework for Perceptually Guided Audio Texture Generation

TL;DR

This paper introduces an example-based method to control audio texture generation using StyleGAN without labeled datasets, enabling semantic attribute manipulation and transfer in generated audio textures.

Contribution

It presents a novel framework that infers guidance vectors for semantic control in unconditionally trained StyleGANs using minimal synthetic examples.

Findings

Successfully controls semantic attributes in audio textures

Enables semantic attribute transfer between audio samples

Works without large labeled datasets

Abstract

Controllable generation using StyleGANs is usually achieved by training the model using labeled data. For audio textures, however, there is currently a lack of large semantically labeled datasets. Therefore, to control generation, we develop a method for semantic control over an unconditionally trained StyleGAN in the absence of such labeled datasets. In this paper, we propose an example-based framework to determine guidance vectors for audio texture generation based on user-defined semantic attributes. Our approach leverages the semantically disentangled latent space of an unconditionally trained StyleGAN. By using a few synthetic examples to indicate the presence or absence of a semantic attribute, we infer the guidance vectors in the latent space of the StyleGAN to control that attribute during generation. Our results show that our framework can find user-defined and perceptually relevant guidance vectors for controllable generation for audio textures. Furthermore, we demonstrate an application of our framework to other tasks, such as selective semantic attribute transfer.