ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modeling

TL;DR

ActiveRIR introduces a reinforcement learning approach for efficient acoustic environment modeling using audio-visual data, significantly reducing data collection needs while accurately capturing environment acoustics.

Contribution

It presents a novel RL-based active sampling method that guides a mobile agent to construct high-quality acoustic models with minimal acoustic samples in unseen environments.

Findings

ActiveRIR outperforms traditional and state-of-the-art methods in diverse environments.

The RL policy effectively leverages audio-visual information for navigation and sampling.

High-quality acoustic models are achieved with fewer samples than existing approaches.

Abstract

An environment acoustic model represents how sound is transformed by the physical characteristics of an indoor environment, for any given source/receiver location. Traditional methods for constructing acoustic models involve expensive and time-consuming collection of large quantities of acoustic data at dense spatial locations in the space, or rely on privileged knowledge of scene geometry to intelligently select acoustic data sampling locations. We propose active acoustic sampling, a new task for efficiently building an environment acoustic model of an unmapped environment in which a mobile agent equipped with visual and acoustic sensors jointly constructs the environment acoustic model and the occupancy map on-the-fly. We introduce ActiveRIR, a reinforcement learning (RL) policy that leverages information from audio-visual sensor streams to guide agent navigation and determine optimal acoustic data sampling positions, yielding a high quality acoustic model of the environment from a minimal set of acoustic samples. We train our policy with a novel RL reward based on information gain in the environment acoustic model. Evaluating on diverse unseen indoor environments from a state-of-the-art acoustic simulation platform, ActiveRIR outperforms an array of methods--both traditional navigation agents based on spatial novelty and visual exploration as well as existing state-of-the-art methods.