Reverberation: Learning the Latencies Before Forecasting Trajectories

TL;DR

This paper introduces a novel reverberation transform and model for trajectory prediction that explicitly learns and models agent-specific latencies, improving the realism and interpretability of predicted trajectories.

Contribution

It proposes a new reverberation-based approach to explicitly learn and incorporate agent latencies into trajectory prediction models, addressing a key challenge in causal continuity.

Findings

Achieves competitive accuracy on multiple datasets.

Reveals interpretable latency dynamics across agents and scenarios.

Demonstrates potential as a general latency modeling approach.

Abstract

Bridging the past to the future, connecting agents both spatially and temporally, lies at the core of the trajectory prediction task. Despite great efforts, it remains challenging to explicitly learn and predict latencies, i.e., response intervals or temporal delays with which agents respond to various trajectory-changing events and adjust their future paths, whether on their own or interactively. Different agents may exhibit distinct latency preferences for noticing, processing, and reacting to a specific trajectory-changing event. The lack of consideration of such latencies may undermine the causal continuity of forecasting systems, leading to implausible or unintended trajectories. Inspired by reverberation in acoustics, we propose a new reverberation transform and the corresponding Reverberation (short for Rev) trajectory prediction model, which predicts both individual latency preferences and their stochastic variations accordingly, by using two explicit and learnable reverberation kernels, enabling latency-conditioned and controllable trajectory prediction of both non-interactive and social latencies. Experiments on multiple datasets, whether pedestrians or vehicles, demonstrate that Rev achieves competitive accuracy while revealing interpretable latency dynamics across agents and scenarios. Qualitative analyses further verify the properties of the reverberation transform, highlighting its potential as a general latency modeling approach.