Deep Impulse Responses: Estimating and Parameterizing Filters with Deep Networks

TL;DR

This paper introduces a neural network-based framework for estimating and parameterizing impulse responses from noisy, real-world data, enabling robust, efficient, and interpolatable impulse response modeling for AR/VR applications.

Contribution

A novel neural representation learning framework that jointly estimates impulse responses and noise characteristics from challenging, real-world signals.

Findings

Robust impulse response estimation at low SNRs

Effective learning from spatio-temporal speech data

Facilitates interpolation and compression of impulse responses

Abstract

Impulse response estimation in high noise and in-the-wild settings, with minimal control of the underlying data distributions, is a challenging problem. We propose a novel framework for parameterizing and estimating impulse responses based on recent advances in neural representation learning. Our framework is driven by a carefully designed neural network that jointly estimates the impulse response and the (apriori unknown) spectral noise characteristics of an observed signal given the source signal. We demonstrate robustness in estimation, even under low signal-to-noise ratios, and show strong results when learning from spatio-temporal real-world speech data. Our framework provides a natural way to interpolate impulse responses on a spatial grid, while also allowing for efficiently compressing and storing them for real-time rendering applications in augmented and virtual reality.