# Learning a Memory-Enhanced Multi-Stage Goal-Driven Network for Egocentric Trajectory Prediction

## TL;DR

The paper introduces a new network for predicting trajectories in dynamic scenes by using memory of past experiences and goal-driven stages.

## Contribution

The novel contribution is a memory-enhanced, goal-driven network that uses scene layout memory and multi-stage goals for trajectory prediction.

## Key findings

- The proposed ME-MGNet outperforms existing methods on three public datasets and a new Fuzhou DashCam dataset.
- Scene layout memory improves trajectory prediction by leveraging prior experiences in visually similar scenes.
- Multi-stage goal generation and memory filtering enhance prediction accuracy in dynamic environments.

## Abstract

We propose a memory-enhanced multi-stage goal-driven network (ME-MGNet) for egocentric trajectory prediction in dynamic scenes. Our key idea is to build a scene layout memory inspired by human perception in order to transfer knowledge from prior experiences to the current scenario in a top-down manner. Specifically, given a test scene, we first perform scene-level matching based on our scene layout memory to retrieve trajectories from visually similar scenes in the training data. This is followed by trajectory-level matching and memory filtering to obtain a set of goal features. In addition, a multi-stage goal generator takes these goal features and uses a backward decoder to produce several stage goals. Finally, we integrate the above steps into a conditional autoencoder and a forward decoder to produce trajectory prediction results. Experiments on three public datasets, JAAD, PIE, and KITTI, and a new egocentric trajectory prediction dataset, Fuzhou DashCam (FZDC), validate the efficacy of the proposed method.