The Un-Kidnappable Robot: Acoustic Localization of Sneaking People

TL;DR

This paper explores passive audio-based localization to detect and track quietly moving people around robots, demonstrating effective detection and tracking in indoor environments using only incidental sounds.

Contribution

It introduces a novel dataset and models for acoustic localization of sneaking individuals, enabling robots to detect and track quiet movements without active sensing.

Findings

Models accurately detect presence of moving people using audio

Robots successfully track quiet-moving individuals in real-time

Passive audio sensing can be effective for stealthy person detection

Abstract

How easy is it to sneak up on a robot? We examine whether we can detect people using only the incidental sounds they produce as they move, even when they try to be quiet. We collect a robotic dataset of high-quality 4-channel audio paired with 360 degree RGB data of people moving in different indoor settings. We train models that predict if there is a moving person nearby and their location using only audio. We implement our method on a robot, allowing it to track a single person moving quietly with only passive audio sensing. For demonstration videos, see our project page: https://sites.google.com/view/unkidnappable-robot