Learning Continuous Representation of Audio for Arbitrary Scale Super Resolution

TL;DR

This paper introduces LISA, a novel implicit neural representation method that models audio as a continuous function, enabling super resolution at arbitrary scales and surpassing fixed-scale methods in performance.

Contribution

LISA provides a continuous, local implicit representation of audio, allowing super resolution at any scale without retraining for each resolution, which is a significant advancement over fixed-scale approaches.

Findings

LISA outperforms previous fixed-scale methods in super resolution quality.

LISA requires fewer parameters than traditional methods.

LISA can perform super resolution beyond the original training resolution.

Abstract

Audio super resolution aims to predict the missing high resolution components of the low resolution audio signals. While audio in nature is a continuous signal, current approaches treat it as discrete data (i.e., input is defined on discrete time domain), and consider the super resolution over a fixed scale factor (i.e., it is required to train a new neural network to change output resolution). To obtain a continuous representation of audio and enable super resolution for arbitrary scale factor, we propose a method of implicit neural representation, coined Local Implicit representation for Super resolution of Arbitrary scale (LISA). Our method locally parameterizes a chunk of audio as a function of continuous time, and represents each chunk with the local latent codes of neighboring chunks so that the function can extrapolate the signal at any time coordinate, i.e., infinite resolution. To learn a continuous representation for audio, we design a self-supervised learning strategy to practice super resolution tasks up to the original resolution by stochastic selection. Our numerical evaluation shows that LISA outperforms the previous fixed-scale methods with a fraction of parameters, but also is capable of arbitrary scale super resolution even beyond the resolution of training data.