Reflective Teacher: Semi-Supervised Multimodal 3D Object Detection in Bird's-Eye-View via Uncertainty Measure

TL;DR

This paper introduces a Reflective Teacher framework for semi-supervised 3D object detection in Bird's-Eye-View, combining a knowledge retention mechanism with a geometry-aware multimodal fusion technique, leading to improved performance with limited labeled data.

Contribution

It proposes a novel Reflective Teacher approach that mitigates catastrophic forgetting and a Geometry Aware BEV Fusion method for better multi-modal feature alignment.

Findings

Achieves state-of-the-art performance in semi-supervised 3D detection

Uses only 22-25% of labeled data to match fully supervised results

Demonstrates effectiveness on nuScenes and Waymo datasets

Abstract

Applying pseudo labeling techniques has been found to be advantageous in semi-supervised 3D object detection (SSOD) in Bird's-Eye-View (BEV) for autonomous driving, particularly where labeled data is limited. In the literature, Exponential Moving Average (EMA) has been used for adjustments of the weights of teacher network by the student network. However, the same induces catastrophic forgetting in the teacher network. In this work, we address this issue by introducing a novel concept of Reflective Teacher where the student is trained by both labeled and pseudo labeled data while its knowledge is progressively passed to the teacher through a regularizer to ensure retention of previous knowledge. Additionally, we propose Geometry Aware BEV Fusion (GA-BEVFusion) for efficient alignment of multi-modal BEV features, thus reducing the disparity between the modalities - camera and LiDAR. This helps to map the precise geometric information embedded among LiDAR points reliably with the spatial priors for extraction of semantic information from camera images. Our experiments on the nuScenes and Waymo datasets demonstrate: 1) improved performance over state-of-the-art methods in both fully supervised and semi-supervised settings; 2) Reflective Teacher achieves equivalent performance with only 25% and 22% of labeled data for nuScenes and Waymo datasets respectively, in contrast to other fully supervised methods that utilize the full labeled dataset.