StereoDistill: Pick the Cream from LiDAR for Distilling Stereo-based 3D Object Detection

TL;DR

StereoDistill introduces a novel cross-modal distillation approach that effectively transfers knowledge from LiDAR-based 3D object detection models to stereo-based models, significantly improving stereo detection performance.

Contribution

The paper proposes StereoDistill, a new distillation method with XGD and CLD techniques to enhance stereo 3D detection by leveraging LiDAR models, addressing overlooked response-level distillation.

Findings

Achieves state-of-the-art stereo 3D detection results on KITTI.

Effectively narrows the gap between stereo and LiDAR-based detection.

Validates the approach on KITTI and Argoverse datasets.

Abstract

In this paper, we propose a cross-modal distillation method named StereoDistill to narrow the gap between the stereo and LiDAR-based approaches via distilling the stereo detectors from the superior LiDAR model at the response level, which is usually overlooked in 3D object detection distillation. The key designs of StereoDistill are: the X-component Guided Distillation~(XGD) for regression and the Cross-anchor Logit Distillation~(CLD) for classification. In XGD, instead of empirically adopting a threshold to select the high-quality teacher predictions as soft targets, we decompose the predicted 3D box into sub-components and retain the corresponding part for distillation if the teacher component pilot is consistent with ground truth to largely boost the number of positive predictions and alleviate the mimicking difficulty of the student model. For CLD, we aggregate the probability distribution of all anchors at the same position to encourage the highest probability anchor rather than individually distill the distribution at the anchor level. Finally, our StereoDistill achieves state-of-the-art results for stereo-based 3D detection on the KITTI test benchmark and extensive experiments on KITTI and Argoverse Dataset validate the effectiveness.