HAMF: A Hybrid Attention-Mamba Framework for Joint Scene Context Understanding and Future Motion Representation Learning

TL;DR

HAMF introduces a hybrid attention and Mamba-based framework that jointly models scene context and future motion for improved autonomous driving trajectory prediction, achieving state-of-the-art results.

Contribution

The paper presents a novel joint scene understanding and motion prediction framework using attention mechanisms and a Mamba module, enhancing accuracy and diversity in forecasts.

Findings

Achieves state-of-the-art performance on Argoverse 2 benchmark.

Effectively combines self-attention and cross-attention for scene and motion modeling.

Demonstrates lightweight architecture with high accuracy.

Abstract

Motion forecasting represents a critical challenge in autonomous driving systems, requiring accurate prediction of surrounding agents' future trajectories. While existing approaches predict future motion states with the extracted scene context feature from historical agent trajectories and road layouts, they suffer from the information degradation during the scene feature encoding. To address the limitation, we propose HAMF, a novel motion forecasting framework that learns future motion representations with the scene context encoding jointly, to coherently combine the scene understanding and future motion state prediction. We first embed the observed agent states and map information into 1D token sequences, together with the target multi-modal future motion features as a set of learnable tokens. Then we design a unified Attention-based encoder, which synergistically combines self-attention and cross-attention mechanisms to model the scene context information and aggregate future motion features jointly. Complementing the encoder, we implement the Mamba module in the decoding stage to further preserve the consistency and correlations among the learned future motion representations, to generate the accurate and diverse final trajectories. Extensive experiments on Argoverse 2 benchmark demonstrate that our hybrid Attention-Mamba model achieves state-of-the-art motion forecasting performance with the simple and lightweight architecture.