Auditory System for a Mobile Robot

TL;DR

This thesis presents an artificial auditory system for mobile robots capable of localizing, tracking, separating, and recognizing multiple sound sources in noisy environments, enhancing human-robot interaction.

Contribution

It introduces a non-human-inspired microphone array system with integrated algorithms for sound source localization, tracking, separation, and speech recognition, demonstrating real-time multi-source processing.

Findings

Tracks up to four sound sources in noisy, reverberant environments

Achieves 13.7 dB SNR improvement with three speakers

Over 80% digit recognition accuracy in tested scenarios

Abstract

In this thesis, we propose an artificial auditory system that gives a robot the ability to locate and track sounds, as well as to separate simultaneous sound sources and recognising simultaneous speech. We demonstrate that it is possible to implement these capabilities using an array of microphones, without trying to imitate the human auditory system. The sound source localisation and tracking algorithm uses a steered beamformer to locate sources, which are then tracked using a multi-source particle filter. Separation of simultaneous sound sources is achieved using a variant of the Geometric Source Separation (GSS) algorithm, combined with a multi-source post-filter that further reduces noise, interference and reverberation. Speech recognition is performed on separated sources, either directly or by using Missing Feature Theory (MFT) to estimate the reliability of the speech features. The results obtained show that it is possible to track up to four simultaneous sound sources, even in noisy and reverberant environments. Real-time control of the robot following a sound source is also demonstrated. The sound source separation approach we propose is able to achieve a 13.7 dB improvement in signal-to-noise ratio compared to a single microphone when three speakers are present. In these conditions, the system demonstrates more than 80% accuracy on digit recognition, higher than most human listeners could obtain in our small case study when recognising only one of these sources. All these new capabilities will allow humans to interact more naturally with a mobile robot in real life settings.