Deep Room Impulse Response Completion

TL;DR

This paper introduces RIR completion, a neural network task that synthesizes late reverberation in room impulse responses from early sound data, enabling faster and more efficient spatial audio rendering in VR and gaming.

Contribution

It proposes DECOR, a deep neural network for predicting late reverberation from early RIR segments, a novel approach to improve RIR generation efficiency.

Findings

DECOR achieves comparable performance to state-of-the-art models.

The method demonstrates feasibility for fast late reverberation synthesis.

Supports integration with various spatial audio rendering techniques.

Abstract

Rendering immersive spatial audio in virtual reality (VR) and video games demands a fast and accurate generation of room impulse responses (RIRs) to recreate auditory environments plausibly. However, the conventional methods for simulating or measuring long RIRs are either computationally intensive or challenged by low signal-to-noise ratios. This study is propelled by the insight that direct sound and early reflections encapsulate sufficient information about room geometry and absorption characteristics. Building upon this premise, we propose a novel task termed "RIR completion," aimed at synthesizing the late reverberation given only the early portion (50 ms) of the response. To this end, we introduce DECOR, Deep Exponential Completion Of Room impulse responses, a deep neural network structured as an autoencoder designed to predict multi-exponential decay envelopes of filtered noise sequences. The interpretability of DECOR's output facilitates its integration with diverse rendering techniques. The proposed method is compared against an adapted state-of-the-art network, and comparable performance shows promising results supporting the feasibility of the RIR completion task. The RIR completion can be widely adapted to enhance RIR generation tasks where fast late reverberation approximation is required.