Remember and Recall: Associative-Memory-based Trajectory Prediction

TL;DR

This paper introduces FMTP, a trajectory prediction model inspired by human memory, using discrete representations and an advanced reasoning engine to improve efficiency and performance in autonomous driving scenarios.

Contribution

The paper proposes a novel fragmented-memory-based model that employs discrete trajectory representations and a reasoning engine, enhancing efficiency and experience utilization in trajectory prediction.

Findings

Achieves superior performance on multiple public datasets.

Reduces computational redundancy with discrete representations.

Effectively leverages past experiences for current trajectory prediction.

Abstract

Trajectory prediction is a pivotal component of autonomous driving systems, enabling the application of accumulated movement experience to current scenarios. Although most existing methods concentrate on learning continuous representations to gain valuable experience, they often suffer from computational inefficiencies and struggle with unfamiliar situations. To address this issue, we propose the Fragmented-Memory-based Trajectory Prediction (FMTP) model, inspired by the remarkable learning capabilities of humans, particularly their ability to leverage accumulated experience and recall relevant memories in unfamiliar situations. The FMTP model employs discrete representations to enhance computational efficiency by reducing information redundancy while maintaining the flexibility to utilize past experiences. Specifically, we design a learnable memory array by consolidating continuous trajectory representations from the training set using defined quantization operations during the training phase. This approach further eliminates redundant information while preserving essential features in discrete form. Additionally, we develop an advanced reasoning engine based on language models to deeply learn the associative rules among these discrete representations. Our method has been evaluated on various public datasets, including ETH-UCY, inD, SDD, nuScenes, Waymo, and VTL-TP. The extensive experimental results demonstrate that our approach achieves significant performance and extracts more valuable experience from past trajectories to inform the current state.