LiDAR-Based 3D Object Detection via Hybrid 2D Semantic Scene Generation

TL;DR

This paper introduces a hybrid 2D semantic scene generation method that enhances LiDAR-based 3D object detection by providing dense supervision signals to improve BEV feature learning, leading to better proposal generation.

Contribution

It proposes a novel scene representation encoding semantics and geometry in 2D, with auxiliary networks predicting semantic probabilities to improve BEV features in 3D detectors.

Findings

Improves baseline 3D detectors with dense semantic supervision.

Easily integrable into existing state-of-the-art models.

Consistently enhances proposal generation performance.

Abstract

Bird's-Eye View (BEV) features are popular intermediate scene representations shared by the 3D backbone and the detector head in LiDAR-based object detectors. However, little research has been done to investigate how to incorporate additional supervision on the BEV features to improve proposal generation in the detector head, while still balancing the number of powerful 3D layers and efficient 2D network operations. This paper proposes a novel scene representation that encodes both the semantics and geometry of the 3D environment in 2D, which serves as a dense supervision signal for better BEV feature learning. The key idea is to use auxiliary networks to predict a combination of explicit and implicit semantic probabilities by exploiting their complementary properties. Extensive experiments show that our simple yet effective design can be easily integrated into most state-of-the-art 3D object detectors and consistently improves upon baseline models.