NDF+: Joint Neural Directional Filtering and Diffuse Sound Extraction

TL;DR

NDF+ extends neural directional filtering to jointly reconstruct virtual directional microphones and extract diffuse sounds, enabling diffuse sound control and outperforming traditional methods in reverberant environments.

Contribution

NDF+ introduces a joint approach for neural directional filtering and diffuse sound extraction, allowing diffuse component manipulation in VDM reconstruction.

Findings

NDF+ outperforms baseline methods on both VDM reconstruction and diffuse sound extraction.

NDF+ maintains comparable VDM quality to single-task NDF while enabling diffuse sound control.

In stereo recordings, NDF+ allows controllable inter-channel level differences.

Abstract

Recently, neural directional filtering (NDF) has been introduced as a flexible approach for reconstructing a virtual directional microphone (VDM) with a desired directivity pattern for spatial sound capture. Building on this idea, we propose NDF+, which enables joint neural directional filtering and diffuse sound extraction. NDF+ reformulates VDM estimation into two coupled subtasks: dereverberated VDM reconstruction and diffuse sound extraction. This reformulation enables NDF+ to manipulate diffuse components in the final reconstructed VDM output. We evaluated NDF+ under reverberant conditions and compared it with representative conventional baselines. Results show that NDF+ consistently outperforms the baselines on both subtasks, while maintaining VDM reconstruction quality comparable to that of the original single-task NDF model. These findings indicate that NDF+ introduces an additional degree of freedom for diffuse sound control in the VDM reconstruction. In a stereo recording application, NDF+ provides controllable inter-channel level differences between left and right channels by adjusting the estimated diffuse component.