Differentiable Semantic Meta-Learning Framework for Long-Tail Motion Forecasting in Autonomous Driving

TL;DR

This paper introduces SAML, a novel differentiable meta-learning framework that models the rarity of motion events in autonomous driving using semantic properties, enabling rapid adaptation and improved safety in long-tail scenarios.

Contribution

SAML is the first framework to define a differentiable tailness measure for motion forecasting, combining semantic properties with meta-learning for fast adaptation to rare events.

Findings

Achieves state-of-the-art accuracy on benchmark datasets.

Significantly improves worst-case event prediction performance.

Maintains high computational efficiency.

Abstract

Long-tail motion forecasting is a core challenge for autonomous driving, where rare yet safety-critical events-such as abrupt maneuvers and dense multi-agent interactions-dominate real-world risk. Existing approaches struggle in these scenarios because they rely on either non-interpretable clustering or model-dependent error heuristics, providing neither a differentiable notion of "tailness" nor a mechanism for rapid adaptation. We propose SAML, a Semantic-Aware Meta-Learning framework that introduces the first differentiable definition of tailness for motion forecasting. SAML quantifies motion rarity via semantically meaningful intrinsic (kinematic, geometric, temporal) and interactive (local and global risk) properties, which are fused by a Bayesian Tail Perceiver into a continuous, uncertainty-aware Tail Index. This Tail Index drives a meta-memory adaptation module that couples a dynamic prototype memory with an MAML-based cognitive set mechanism, enabling fast adaptation to rare or evolving patterns. Experiments on nuScenes, NGSIM, and HighD show that SAML achieves state-of-the-art overall accuracy and substantial gains on top 1-5% worst-case events, while maintaining high efficiency. Our findings highlight semantic meta-learning as a pathway toward robust and safety-critical motion forecasting.