DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation

TL;DR

DV-3DLane introduces an end-to-end multi-modal 3D lane detection framework that combines image and LiDAR data in dual-view spaces, significantly improving accuracy and robustness for autonomous driving applications.

Contribution

The paper proposes a novel dual-view multi-modal framework with bidirectional feature fusion, unified query generation, and dual-view deformable attention for enhanced 3D lane detection.

Findings

Achieves state-of-the-art F1 score with 11.2 gain on OpenLane benchmark.

Reduces detection errors by 53.5%.

Demonstrates effective multi-modal feature integration in dual-view spaces.

Abstract

Accurate 3D lane estimation is crucial for ensuring safety in autonomous driving. However, prevailing monocular techniques suffer from depth loss and lighting variations, hampering accurate 3D lane detection. In contrast, LiDAR points offer geometric cues and enable precise localization. In this paper, we present DV-3DLane, a novel end-to-end Dual-View multi-modal 3D Lane detection framework that synergizes the strengths of both images and LiDAR points. We propose to learn multi-modal features in dual-view spaces, i.e., perspective view (PV) and bird's-eye-view (BEV), effectively leveraging the modal-specific information. To achieve this, we introduce three designs: 1) A bidirectional feature fusion strategy that integrates multi-modal features into each view space, exploiting their unique strengths. 2) A unified query generation approach that leverages lane-aware knowledge from both PV and BEV spaces to generate queries. 3) A 3D dual-view deformable attention mechanism, which aggregates discriminative features from both PV and BEV spaces into queries for accurate 3D lane detection. Extensive experiments on the public benchmark, OpenLane, demonstrate the efficacy and efficiency of DV-3DLane. It achieves state-of-the-art performance, with a remarkable 11.2 gain in F1 score and a substantial 53.5% reduction in errors. The code is available at \url{https://github.com/JMoonr/dv-3dlane}.