Deep Detection of People and their Mobility Aids for a Hospital Robot

TL;DR

This paper presents a fast, depth-based perception system for hospital robots to detect and track people and their mobility aids, improving safety and assistance in complex environments.

Contribution

It introduces a novel perception pipeline with a rapid region proposal method and probabilistic estimation for accurate tracking of people and mobility aids.

Findings

Sevenfold speed-up in object detection compared to traditional methods

Successful tracking of multiple people with occlusions and different mobility aids

Introduction of a new annotated hospital RGB-D dataset with over 17,000 images

Abstract

Robots operating in populated environments encounter many different types of people, some of whom might have an advanced need for cautious interaction, because of physical impairments or their advanced age. Robots therefore need to recognize such advanced demands to provide appropriate assistance, guidance or other forms of support. In this paper, we propose a depth-based perception pipeline that estimates the position and velocity of people in the environment and categorizes them according to the mobility aids they use: pedestrian, person in wheelchair, person in a wheelchair with a person pushing them, person with crutches and person using a walker. We present a fast region proposal method that feeds a Region-based Convolutional Network (Fast R-CNN). With this, we speed up the object detection process by a factor of seven compared to a dense sliding window approach. We furthermore propose a probabilistic position, velocity and class estimator to smooth the CNN's detections and account for occlusions and misclassifications. In addition, we introduce a new hospital dataset with over 17,000 annotated RGB-D images. Extensive experiments confirm that our pipeline successfully keeps track of people and their mobility aids, even in challenging situations with multiple people from different categories and frequent occlusions. Videos of our experiments and the dataset are available at http://www2.informatik.uni-freiburg.de/~kollmitz/MobilityAids