TBD Pedestrian Data Collection: Towards Rich, Portable, and Large-Scale Natural Pedestrian Data

TL;DR

This paper introduces a portable, large-scale pedestrian data collection system with a semi-autonomous labeling pipeline, enabling rich, verified datasets for social navigation research in diverse environments.

Contribution

The paper presents a novel data collection system combining multiple views, natural pedestrian behavior, and human-verified labels, creating a comprehensive dataset for pedestrian behavior modeling.

Findings

Collected a larger, richer dataset than prior works.

System enables fast, scalable data collection with human verification.

Supports new research opportunities in social navigation.

Abstract

Social navigation and pedestrian behavior research has shifted towards machine learning-based methods and converged on the topic of modeling inter-pedestrian interactions and pedestrian-robot interactions. For this, large-scale datasets that contain rich information are needed. We describe a portable data collection system, coupled with a semi-autonomous labeling pipeline. As part of the pipeline, we designed a label correction web app that facilitates human verification of automated pedestrian tracking outcomes. Our system enables large-scale data collection in diverse environments and fast trajectory label production. Compared with existing pedestrian data collection methods, our system contains three components: a combination of top-down and ego-centric views, natural human behavior in the presence of a socially appropriate "robot", and human-verified labels grounded in the metric space. To the best of our knowledge, no prior data collection system has a combination of all three components. We further introduce our ever-expanding dataset from the ongoing data collection effort -- the TBD Pedestrian Dataset and show that our collected data is larger in scale, contains richer information when compared to prior datasets with human-verified labels, and supports new research opportunities.