LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search

TL;DR

LVNS-RAVE combines evolutionary algorithms with deep learning to generate diverse, realistic, and novel sounds, offering controllability and potential as a creative tool for artists.

Contribution

It introduces a novel method integrating LVNS with RAVE and VGGish models to enhance sound diversity and novelty in audio generation.

Findings

Successfully generates diversified, novel audio samples.

Generation process is controllable via mutation parameters.

Effective across different pre-trained RAVE models.

Abstract

Evolutionary Algorithms and Generative Deep Learning have been two of the most powerful tools for sound generation tasks. However, they have limitations: Evolutionary Algorithms require complicated designs, posing challenges in control and achieving realistic sound generation. Generative Deep Learning models often copy from the dataset and lack creativity. In this paper, we propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic and novel sounds. We use the RAVE model as the sound generator and the VGGish model as a novelty evaluator in the Latent Vector Novelty Search (LVNS) algorithm. The reported experiments show that the method can successfully generate diversified, novel audio samples under different mutation setups using different pre-trained RAVE models. The characteristics of the generation process can be easily controlled with the mutation parameters. The proposed algorithm can be a creative tool for sound artists and musicians.