HiMAP: History-aware Map-occupancy Prediction with Fallback

TL;DR

HiMAP is a novel, tracking-free motion prediction framework for autonomous driving that remains reliable during tracking failures by using historical occupancy maps and a query-based decoder, achieving competitive results without relying on object identities.

Contribution

HiMAP introduces a history-aware, tracking-free prediction method that leverages occupancy maps and a query module to improve robustness during tracking failures in autonomous driving.

Findings

Achieves comparable performance to tracking-based methods on Argoverse 2.

Outperforms baselines in no-tracking scenarios with significant metric improvements.

Provides stable, multi-agent forecasts without relying on object identities.

Abstract

Accurate motion forecasting is critical for autonomous driving, yet most predictors rely on multi-object tracking (MOT) with identity association, assuming that objects are correctly and continuously tracked. When tracking fails due to, e.g., occlusion, identity switches, or missed detections, prediction quality degrades and safety risks increase. We present \textbf{HiMAP}, a tracking-free, trajectory prediction framework that remains reliable under MOT failures. HiMAP converts past detections into spatiotemporally invariant historical occupancy maps and introduces a historical query module that conditions on the current agent state to iteratively retrieve agent-specific history from unlabeled occupancy representations. The retrieved history is summarized by a temporal map embedding and, together with the final query and map context, drives a DETR-style decoder to produce multi-modal future trajectories. This design lifts identity reliance, supports streaming inference via reusable encodings, and serves as a robust fallback when tracking is unavailable. On Argoverse~2, HiMAP achieves performance comparable to tracking-based methods while operating without IDs, and it substantially outperforms strong baselines in the no-tracking setting, yielding relative gains of 11\% in FDE, 12\% in ADE, and a 4\% reduction in MR over a fine-tuned QCNet. Beyond aggregate metrics, HiMAP delivers stable forecasts for all agents simultaneously without waiting for tracking to recover, highlighting its practical value for safety-critical autonomy. The code is available under: https://github.com/XuYiMing83/HiMAP.