Lend me an Ear: Speech Enhancement Using a Robotic Arm with a Microphone Array

TL;DR

This paper presents a novel speech enhancement method using a robotic arm-mounted microphone array that dynamically reconfigures to improve signal quality in noisy environments, outperforming traditional static setups.

Contribution

The study introduces a physical optimization approach combining robotic manipulation, sound localization, and deep learning to adapt microphone array geometry for enhanced speech clarity.

Findings

Outperforms traditional configurations in signal-to-distortion ratio

Achieves lower word error rate across various noise conditions

Demonstrates effective integration of robotics and signal processing techniques

Abstract

Speech enhancement performance degrades significantly in noisy environments, limiting the deployment of speech-controlled technologies in industrial settings, such as manufacturing plants. Existing speech enhancement solutions primarly rely on advanced digital signal processing techniques, deep learning methods, or complex software optimization techniques. This paper introduces a novel enhancement strategy that incorporates a physical optimization stage by dynamically modifying the geometry of a microphone array to adapt to changing acoustic conditions. A sixteen-microphone array is mounted on a robotic arm manipulator with seven degrees of freedom, with microphones divided into four groups of four, including one group positioned near the end-effector. The system reconfigures the array by adjusting the manipulator joint angles to place the end-effector microphones closer to the target speaker, thereby improving the reference signal quality. This proposed method integrates sound source localization techniques, computer vision, inverse kinematics, minimum variance distortionless response beamformer and time-frequency masking using a deep neural network. Experimental results demonstrate that this approach outperforms other traditional recording configruations, achieving higher scale-invariant signal-to-distortion ratio and lower word error rate accross multiple input signal-to-noise ratio conditions.