AERO: Audio Super Resolution in the Spectral Domain

TL;DR

AERO introduces a spectral domain audio super-resolution model that uses a complex-valued spectrogram approach with an encoder-decoder architecture, achieving superior results in speech and music quality enhancement.

Contribution

The paper presents a novel spectral domain super-resolution method operating on complex spectrograms with full frequency prediction and combined loss functions, improving over prior low-high frequency concatenation techniques.

Findings

Outperforms baselines in Log-Spectral Distance and ViSQOL

Achieves high quality in speech and music sample rate upscaling

Demonstrates effectiveness across diverse audio types

Abstract

We present AERO, a audio super-resolution model that processes speech and music signals in the spectral domain. AERO is based on an encoder-decoder architecture with U-Net like skip connections. We optimize the model using both time and frequency domain loss functions. Specifically, we consider a set of reconstruction losses together with perceptual ones in the form of adversarial and feature discriminator loss functions. To better handle phase information the proposed method operates over the complex-valued spectrogram using two separate channels. Unlike prior work which mainly considers low and high frequency concatenation for audio super-resolution, the proposed method directly predicts the full frequency range. We demonstrate high performance across a wide range of sample rates considering both speech and music. AERO outperforms the evaluated baselines considering Log-Spectral Distance, ViSQOL, and the subjective MUSHRA test. Audio samples and code are available at https://pages.cs.huji.ac.il/adiyoss-lab/aero