Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty

TL;DR

This paper introduces HAICU, a trajectory forecasting method that explicitly uses class probability distributions of agents, improving prediction accuracy under uncertainty, and presents a new dataset PUP for evaluating perceptual uncertainty in autonomous driving.

Contribution

The paper proposes HAICU, a novel trajectory forecasting approach that incorporates class probabilities, and introduces PUP, a challenging dataset for studying perceptual uncertainty in autonomous driving.

Findings

Incorporating class probabilities improves forecasting accuracy.

HAICU enables counterfactual trajectory predictions.

PUP dataset reflects real-world perception uncertainties.

Abstract

Reasoning about the future behavior of other agents is critical to safe robot navigation. The multiplicity of plausible futures is further amplified by the uncertainty inherent to agent state estimation from data, including positions, velocities, and semantic class. Forecasting methods, however, typically neglect class uncertainty, conditioning instead only on the agent's most likely class, even though perception models often return full class distributions. To exploit this information, we present HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities. We additionally present PUP, a new challenging real-world autonomous driving dataset, to investigate the impact of Perceptual Uncertainty in Prediction. It contains challenging crowded scenes with unfiltered agent class probabilities that reflect the long-tail of current state-of-the-art perception systems. We demonstrate that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty, and enables new forecasting capabilities such as counterfactual predictions.