VectorMapNet: End-to-end Vectorized HD Map Learning

TL;DR

VectorMapNet is an end-to-end deep learning pipeline that predicts vectorized HD maps directly from onboard sensor data, improving accuracy and detail over previous raster-based methods for autonomous driving.

Contribution

It introduces the first end-to-end framework for vectorized HD map learning from onboard observations, explicitly modeling spatial relations between map elements.

Findings

Surpasses state-of-the-art by 14.2 mAP on nuScenes

Achieves detailed and comprehensive map generation

Capable of capturing fine-grained road geometry

Abstract

Autonomous driving systems require High-Definition (HD) semantic maps to navigate around urban roads. Existing solutions approach the semantic mapping problem by offline manual annotation, which suffers from serious scalability issues. Recent learning-based methods produce dense rasterized segmentation predictions to construct maps. However, these predictions do not include instance information of individual map elements and require heuristic post-processing to obtain vectorized maps. To tackle these challenges, we introduce an end-to-end vectorized HD map learning pipeline, termed VectorMapNet. VectorMapNet takes onboard sensor observations and predicts a sparse set of polylines in the bird's-eye view. This pipeline can explicitly model the spatial relation between map elements and generate vectorized maps that are friendly to downstream autonomous driving tasks. Extensive experiments show that VectorMapNet achieve strong map learning performance on both nuScenes and Argoverse2 dataset, surpassing previous state-of-the-art methods by 14.2 mAP and 14.6mAP. Qualitatively, VectorMapNet is capable of generating comprehensive maps and capturing fine-grained details of road geometry. To the best of our knowledge, VectorMapNet is the first work designed towards end-to-end vectorized map learning from onboard observations. Our project website is available at \url{https://tsinghua-mars-lab.github.io/vectormapnet/}.