Autonomous In-Situ Soundscape Augmentation via Joint Selection of Masker and Gain

TL;DR

This paper presents a deep learning system for real-time, in-situ soundscape augmentation that jointly selects maskers and gain levels to enhance acoustic comfort, validated on a large subjective response dataset.

Contribution

It introduces a modular deep learning model for joint masker and gain selection, enabling quick inference and feature-domain augmentation, reducing computational costs and pre-calibration needs.

Findings

Model accurately predicts perceptual pleasantness of soundscapes.

System outperforms traditional static selection methods.

Validated on over 440 participants' responses.

Abstract

The selection of maskers and playback gain levels in a soundscape augmentation system is crucial to its effectiveness in improving the overall acoustic comfort of a given environment. Traditionally, the selection of appropriate maskers and gain levels has been informed by expert opinion, which may not representative of the target population, or by listening tests, which can be time-consuming and labour-intensive. Furthermore, the resulting static choices of masker and gain are often inflexible to the dynamic nature of real-world soundscapes. In this work, we utilized a deep learning model to perform joint selection of the optimal masker and its gain level for a given soundscape. The proposed model was designed with highly modular building blocks, allowing for an optimized inference process that can quickly search through a large number of masker and gain combinations. In addition, we introduced the use of feature-domain soundscape augmentation conditioned on the digital gain level, eliminating the computationally expensive waveform-domain mixing process during inference time, as well as the tedious pre-calibration process required for new maskers. The proposed system was validated on a large-scale dataset of subjective responses to augmented soundscapes with more than 440 participants, ensuring the ability of the model to predict combined effect of the masker and its gain level on the perceptual pleasantness level.