SoundSil-DS: Deep Denoising and Segmentation of Sound-field Images with Silhouettes

TL;DR

This paper introduces SoundSil-DS, a deep learning method for denoising and segmenting sound-field images with object silhouettes, improving visualization and analysis of sound interactions in optical sound field imaging.

Contribution

It proposes a novel joint denoising and segmentation model for sound-field images, trained on a new dataset created via acoustic simulation.

Findings

Effective noise removal in simulated and real data

Accurate segmentation of sound fields and object silhouettes

Potential for enhanced 3D sound field reconstruction

Abstract

Development of optical technology has enabled imaging of two-dimensional (2D) sound fields. This acousto-optic sensing enables understanding of the interaction between sound and objects such as reflection and diffraction. Moreover, it is expected to be used an advanced measurement technology for sonars in self-driving vehicles and assistive robots. However, the low sound-pressure sensitivity of the acousto-optic sensing results in high intensity of noise on images. Therefore, denoising is an essential task to visualize and analyze the sound fields. In addition to denoising, segmentation of sound and object silhouette is also required to analyze interactions between them. In this paper, we propose sound-field-images-with-object-silhouette denoising and segmentation (SoundSil-DS) that jointly perform denoising and segmentation for sound fields and object silhouettes on a visualized image. We developed a new model based on the current state-of-the-art denoising network. We also created a dataset to train and evaluate the proposed method through acoustic simulation. The proposed method was evaluated using both simulated and measured data. We confirmed that our method can applied to experimentally measured data. These results suggest that the proposed method may improve the post-processing for sound fields, such as physical model-based three-dimensional reconstruction since it can remove unwanted noise and separate sound fields and other object silhouettes. Our code is available at https://github.com/nttcslab/soundsil-ds.