KEMP: Keyframe-Based Hierarchical End-to-End Deep Model for Long-Term Trajectory Prediction

TL;DR

KEMP is a hierarchical deep learning model that predicts long-term trajectories by automatically learning keyframes, eliminating the need for complex goal-selection algorithms, and achieves top performance on public benchmarks.

Contribution

The paper introduces KEMP, a novel end-to-end framework that uses keyframes for trajectory prediction, simplifying the process and improving accuracy over goal-conditioned methods.

Findings

Ranked 1st on Waymo Open Motion Dataset Leaderboard

Automatically learns keyframes without hand-crafted goal algorithms

Outperforms existing goal-based trajectory prediction methods

Abstract

Predicting future trajectories of road agents is a critical task for autonomous driving. Recent goal-based trajectory prediction methods, such as DenseTNT and PECNet, have shown good performance on prediction tasks on public datasets. However, they usually require complicated goal-selection algorithms and optimization. In this work, we propose KEMP, a hierarchical end-to-end deep learning framework for trajectory prediction. At the core of our framework is keyframe-based trajectory prediction, where keyframes are representative states that trace out the general direction of the trajectory. KEMP first predicts keyframes conditioned on the road context, and then fills in intermediate states conditioned on the keyframes and the road context. Under our general framework, goal-conditioned methods are special cases in which the number of keyframes equal to one. Unlike goal-conditioned methods, our keyframe predictor is learned automatically and does not require hand-crafted goal-selection algorithms. We evaluate our model on public benchmarks and our model ranked 1st on Waymo Open Motion Dataset Leaderboard (as of September 1, 2021).