CASPNet++: Joint Multi-Agent Motion Prediction

TL;DR

CASPNet++ advances multi-agent motion prediction for autonomous driving by integrating enhanced scene understanding, multi-modal trajectory outputs, and multi-sensor data fusion, achieving state-of-the-art results in real-time applications.

Contribution

It introduces an improved joint prediction model with enhanced interaction modeling, scene understanding, and multi-modal trajectory outputs, building upon previous CASPNet work.

Findings

Achieves state-of-the-art performance on nuScenes dataset.

Demonstrates real-time deployment in a vehicle.

Effectively fuses diverse environmental sensors.

Abstract

The prediction of road users' future motion is a critical task in supporting advanced driver-assistance systems (ADAS). It plays an even more crucial role for autonomous driving (AD) in enabling the planning and execution of safe driving maneuvers. Based on our previous work, Context-Aware Scene Prediction Network (CASPNet), an improved system, CASPNet++, is proposed. In this work, we focus on further enhancing the interaction modeling and scene understanding to support the joint prediction of all road users in a scene using spatiotemporal grids to model future occupancy. Moreover, an instance-based output head is introduced to provide multi-modal trajectories for agents of interest. In extensive quantitative and qualitative analysis, we demonstrate the scalability of CASPNet++ in utilizing and fusing diverse environmental input sources such as HD maps, Radar detection, and Lidar segmentation. Tested on the urban-focused prediction dataset nuScenes, CASPNet++ reaches state-of-the-art performance. The model has been deployed in a testing vehicle, running in real-time with moderate computational resources.