T2FPV: Dataset and Method for Correcting First-Person View Errors in Pedestrian Trajectory Prediction

TL;DR

This paper introduces T2FPV, a dataset and method for correcting first-person view errors in pedestrian trajectory prediction, improving accuracy by over 10% and supporting egocentric visual data generation from top-down datasets.

Contribution

The paper presents T2FPV, a novel approach to generate first-person view pedestrian datasets and a module CoFE to correct FPV-specific errors, enhancing trajectory prediction accuracy.

Findings

Reduced displacement error by over 10% on average

Created the T2FPV-ETH dataset for egocentric pedestrian data

Provided software tools for dataset generation and error correction

Abstract

Predicting pedestrian motion is essential for developing socially-aware robots that interact in a crowded environment. While the natural visual perspective for a social interaction setting is an egocentric view, the majority of existing work in trajectory prediction therein has been investigated purely in the top-down trajectory space. To support first-person view trajectory prediction research, we present T2FPV, a method for constructing high-fidelity first-person view (FPV) datasets given a real-world, top-down trajectory dataset; we showcase our approach on the ETH/UCY pedestrian dataset to generate the egocentric visual data of all interacting pedestrians, creating the T2FPV-ETH dataset. In this setting, FPV-specific errors arise due to imperfect detection and tracking, occlusions, and field-of-view (FOV) limitations of the camera. To address these errors, we propose CoFE, a module that further refines the imputation of missing data in an end-to-end manner with trajectory forecasting algorithms. Our method reduces the impact of such FPV errors on downstream prediction performance, decreasing displacement error by more than 10% on average. To facilitate research engagement, we release our T2FPV-ETH dataset and software tools.