A Survey of Deep Learning Audio Generation Methods

TL;DR

This survey comprehensively reviews deep learning techniques for audio generation, covering audio representations, architectures, and evaluation metrics, aiming to guide beginners in understanding current state-of-the-art methods and future research directions.

Contribution

It provides a detailed overview of various deep learning architectures and audio representations used in audio generation, highlighting recent developments and evaluation methods.

Findings

Explains fundamental audio representations and recent developments.

Details various deep learning architectures like GANs, Transformers, Diffusion models.

Summarizes evaluation metrics used in audio generation.

Abstract

This article presents a review of typical techniques used in three distinct aspects of deep learning model development for audio generation. In the first part of the article, we provide an explanation of audio representations, beginning with the fundamental audio waveform. We then progress to the frequency domain, with an emphasis on the attributes of human hearing, and finally introduce a relatively recent development. The main part of the article focuses on explaining basic and extended deep learning architecture variants, along with their practical applications in the field of audio generation. The following architectures are addressed: 1) Autoencoders 2) Generative adversarial networks 3) Normalizing flows 4) Transformer networks 5) Diffusion models. Lastly, we will examine four distinct evaluation metrics that are commonly employed in audio generation. This article aims to offer novice readers and beginners in the field a comprehensive understanding of the current state of the art in audio generation methods as well as relevant studies that can be explored for future research.