Bandwidth Extension on Raw Audio via Generative Adversarial Networks

TL;DR

This paper introduces a GAN-based approach for audio super-resolution that leverages unlabeled data through an autoencoder-based loss, achieving significant improvements in audio quality.

Contribution

It develops a novel GAN architecture for audio super-resolution using an autoencoder-based loss, reducing reliance on labeled datasets and expanding GAN applications beyond images.

Findings

Significant improvements in audio quality over previous methods

Effective use of unlabeled data with autoencoder-based feature losses

New architectural components tailored for audio processing

Abstract

Neural network-based methods have recently demonstrated state-of-the-art results on image synthesis and super-resolution tasks, in particular by using variants of generative adversarial networks (GANs) with supervised feature losses. Nevertheless, previous feature loss formulations rely on the availability of large auxiliary classifier networks, and labeled datasets that enable such classifiers to be trained. Furthermore, there has been comparatively little work to explore the applicability of GAN-based methods to domains other than images and video. In this work we explore a GAN-based method for audio processing, and develop a convolutional neural network architecture to perform audio super-resolution. In addition to several new architectural building blocks for audio processing, a key component of our approach is the use of an autoencoder-based loss that enables training in the GAN framework, with feature losses derived from unlabeled data. We explore the impact of our architectural choices, and demonstrate significant improvements over previous works in terms of both objective and perceptual quality.