A Deep Reinforcement Learning Approach to Audio-Based Navigation in a Multi-Speaker Environment

TL;DR

This paper presents a deep reinforcement learning method enabling an autonomous agent to navigate towards a specific speaker in a multi-speaker environment using only raw audio data, demonstrating robustness and effective target identification.

Contribution

It introduces a novel deep reinforcement learning approach for audio-based navigation in multi-speaker settings, a problem previously underexplored in the literature.

Findings

Agent successfully identifies and moves towards target speaker

Robust to pitch shifting of speakers

Effective with limited training data

Abstract

In this work we use deep reinforcement learning to create an autonomous agent that can navigate in a two-dimensional space using only raw auditory sensory information from the environment, a problem that has received very little attention in the reinforcement learning literature. Our experiments show that the agent can successfully identify a particular target speaker among a set of $N$ predefined speakers in a room and move itself towards that speaker, while avoiding collision with other speakers or going outside the room boundaries. The agent is shown to be robust to speaker pitch shifting and it can learn to navigate the environment, even when a limited number of training utterances are available for each speaker.