Multimodal HD Mapping for Intersections by Intelligent Roadside Units

TL;DR

This paper presents a novel multimodal fusion framework using roadside units for high-definition intersection mapping, along with a new annotated dataset, demonstrating improved semantic segmentation accuracy over unimodal methods.

Contribution

The paper introduces a new IRU-based multimodal fusion framework and the RS-seq dataset for enhanced HD map generation at intersections, addressing occlusion issues in vehicle-based mapping.

Findings

Multimodal approach improves mIoU by 4% over image-only methods.

Multimodal approach improves mIoU by 18% over LiDAR-only methods.

Proposed framework outperforms unimodal baselines in semantic segmentation.

Abstract

High-definition (HD) semantic mapping of complex intersections poses significant challenges for traditional vehicle-based approaches due to occlusions and limited perspectives. This paper introduces a novel camera-LiDAR fusion framework that leverages elevated intelligent roadside units (IRUs). Additionally, we present RS-seq, a comprehensive dataset developed through the systematic enhancement and annotation of the V2X-Seq dataset. RS-seq includes precisely labelled camera imagery and LiDAR point clouds collected from roadside installations, along with vectorized maps for seven intersections annotated with detailed features such as lane dividers, pedestrian crossings, and stop lines. This dataset facilitates the systematic investigation of cross-modal complementarity for HD map generation using IRU data. The proposed fusion framework employs a two-stage process that integrates modality-specific feature extraction and cross-modal semantic integration, capitalizing on camera high-resolution texture and precise geometric data from LiDAR. Quantitative evaluations using the RS-seq dataset demonstrate that our multimodal approach consistently surpasses unimodal methods. Specifically, compared to unimodal baselines evaluated on the RS-seq dataset, the multimodal approach improves the mean Intersection-over-Union (mIoU) for semantic segmentation by 4\% over the image-only results and 18\% over the point cloud-only results. This study establishes a baseline methodology for IRU-based HD semantic mapping and provides a valuable dataset for future research in infrastructure-assisted autonomous driving systems.