Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attention

TL;DR

This paper introduces a method to directly utilize internal BEV features in online mapping for autonomous vehicles, significantly improving inference speed and prediction accuracy by integrating mapping and behavior prediction more effectively.

Contribution

It proposes exposing internal BEV features in online map estimation, enabling tighter integration with trajectory forecasting and enhancing performance.

Findings

Up to 73% faster inference speeds.

Up to 29% more accurate predictions.

Effective integration of mapping and behavior prediction.

Abstract

Understanding road geometry is a critical component of the autonomous vehicle (AV) stack. While high-definition (HD) maps can readily provide such information, they suffer from high labeling and maintenance costs. Accordingly, many recent works have proposed methods for estimating HD maps online from sensor data. The vast majority of recent approaches encode multi-camera observations into an intermediate representation, e.g., a bird's eye view (BEV) grid, and produce vector map elements via a decoder. While this architecture is performant, it decimates much of the information encoded in the intermediate representation, preventing downstream tasks (e.g., behavior prediction) from leveraging them. In this work, we propose exposing the rich internal features of online map estimation methods and show how they enable more tightly integrating online mapping with trajectory forecasting. In doing so, we find that directly accessing internal BEV features yields up to 73% faster inference speeds and up to 29% more accurate predictions on the real-world nuScenes dataset.