LABNet: A Lightweight Attentive Beamforming Network for Ad-hoc Multichannel Microphone Invariant Real-Time Speech Enhancement

TL;DR

LABNet is a lightweight, real-time speech enhancement network that effectively handles ad-hoc microphone arrays with minimal computational resources, maintaining microphone invariance and high performance.

Contribution

Introduces LABNet, a novel low-complexity attentive beamforming network that integrates microphone invariance for ad-hoc array speech enhancement.

Findings

Achieves high speech enhancement performance with minimal computational overhead.

Maintains microphone invariance across different array configurations.

Demonstrates potential for real-time edge-device applications.

Abstract

Multichannel speech enhancement (SE) aims to restore clean speech from noisy measurements by leveraging spatiotemporal signal features. In ad-hoc array conditions, microphone invariance (MI) requires systems to handle different microphone numbers and array geometries. From a practical perspective, multichannel recordings inevitably increase the computational burden for edge-device applications, highlighting the necessity of lightweight and efficient deployments. In this work, we propose a lightweight attentive beamforming network (LABNet) to integrate MI in a low-complexity real-time SE system. We design a three-stage framework for efficient intra-channel modeling and inter-channel interaction. A cross-channel attention module is developed to aggregate features from each channel selectively. Experimental results demonstrate our LABNet achieves impressive performance with ultra-light resource overhead while maintaining the MI, indicating great potential for ad-hoc array processing. The code is available:https://github.com/Jokejiangv/LABNet.git