Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs

TL;DR

This paper introduces Dual-CycleGAN, a novel audio super-resolution approach that effectively utilizes unpaired data through domain adaptation and resampling, addressing out-of-domain quality issues without requiring paired training data.

Contribution

The paper presents a dual-cycleGAN framework that decomposes super-resolution into domain adaptation and resampling, enabling high-quality audio super-resolution with unpaired data.

Findings

Outperforms conventional methods on unpaired data

Effectively handles acoustic mismatch

Significantly improves super-resolution quality

Abstract

Neural audio super-resolution models are typically trained on low- and high-resolution audio signal pairs. Although these methods achieve highly accurate super-resolution if the acoustic characteristics of the input data are similar to those of the training data, challenges remain: the models suffer from quality degradation for out-of-domain data, and paired data are required for training. To address these problems, we propose Dual-CycleGAN, a high-quality audio super-resolution method that can utilize unpaired data based on two connected cycle consistent generative adversarial networks (CycleGAN). Our method decomposes the super-resolution method into domain adaptation and resampling processes to handle acoustic mismatch in the unpaired low- and high-resolution signals. The two processes are then jointly optimized within the CycleGAN framework. Experimental results verify that the proposed method significantly outperforms conventional methods when paired data are not available. Code and audio samples are available from https://chomeyama.github.io/DualCycleGAN-Demo/.