Inference-Adaptive Neural Steering for Real-Time Area-Based Sound Source Separation

TL;DR

This paper introduces a neural steering method that dynamically adjusts the target area for sound source separation during inference without retraining, maintaining high performance across various acoustic scenarios.

Contribution

The paper presents a novel inference-adaptive neural steering technique that shifts the target area in multi-microphone sound separation without retraining the neural network.

Findings

Performs well with multiple speakers inside and outside target area

Achieves comparable results to specialized DNNs in DNSMOS and SI-SDR

Operates with negligible additional computational cost

Abstract

We propose a novel Neural Steering technique that adapts the target area of a spatial-aware multi-microphone sound source separation algorithm during inference without the necessity of retraining the deep neural network (DNN). To achieve this, we first train a DNN aiming to retain speech within a target region, defined by an angular span, while suppressing sound sources stemming from other directions. Afterward, a phase shift is applied to the microphone signals, allowing us to shift the center of the target area during inference at negligible additional cost in computational complexity. Further, we show that the proposed approach performs well in a wide variety of acoustic scenarios, including several speakers inside and outside the target area and additional noise. More precisely, the proposed approach performs on par with DNNs trained explicitly for the steered target area in terms of DNSMOS and SI-SDR.