Encore: Conditioning Trajectory Forecasting via Biased Ego Rehearsals

TL;DR

Encore introduces a novel trajectory prediction model that explicitly learns and conditions on biased ego rehearsals to better capture agent subjectivities, improving prediction accuracy and interpretability.

Contribution

The paper proposes a new method that models agent subjectivities as biased rehearsals, conditioning trajectory predictions on these to enhance performance and interpretability.

Findings

Consistent performance improvements across multiple datasets.

Enhanced interpretability of agent subjectivities.

Effective modeling of anisotropic and structured agent behaviors.

Abstract

Learning and representing the subjectivities of agents has become a challenging but crucial problem in the trajectory prediction task. Such subjectivities not only present specific spatial or temporal structures, but also are anisotropic for all interaction participants. Despite great efforts, it remains difficult to explicitly learn and forecast these subjectivities, let alone further modulate models' predictions through a specific ego's subjectivity. Inspired by prefactual thoughts in psychology and relevant theatrical concepts, we interpret such subjectivities in future trajectories as the continuous process from rehearsal to encore. In the rehearsal phase, the proposed ego predictor focuses on how each ego agent learns to derive and direct a set of explicitly biased rehearsal trajectories for all participants in the scene from the short-term observations. Then, these rehearsal trajectories serve as immediate controls to condition final predictions, providing direct yet distinct ego biases for the prediction network to simulate agents' various subjectivities. Experiments across datasets not only demonstrate a consistent improvement in the performance of the proposed \emph{Encore} trajectory prediction model but also provide clear interpretability regarding subjectivities as biased ego rehearsals.