Finetuning Pre-trained Model with Limited Data for LiDAR-based 3D Object Detection by Bridging Domain Gaps

TL;DR

This paper introduces Domain Adaptive Distill-Tuning (DADT), a method for effectively fine-tuning pre-trained LiDAR-based 3D object detection models with limited target domain data, improving accuracy without overfitting.

Contribution

The paper proposes DADT, a novel domain adaptation technique that uses regularizers in a teacher-student framework to adapt pre-trained models with minimal target data.

Findings

DADT significantly improves detection accuracy on Waymo and KITTI datasets.

The method effectively prevents overfitting with only around 100 target LiDAR frames.

Regularizers align object and context representations between models.

Abstract

LiDAR-based 3D object detectors have been largely utilized in various applications, including autonomous vehicles or mobile robots. However, LiDAR-based detectors often fail to adapt well to target domains with different sensor configurations (e.g., types of sensors, spatial resolution, or FOVs) and location shifts. Collecting and annotating datasets in a new setup is commonly required to reduce such gaps, but it is often expensive and time-consuming. Recent studies suggest that pre-trained backbones can be learned in a self-supervised manner with large-scale unlabeled LiDAR frames. However, despite their expressive representations, they remain challenging to generalize well without substantial amounts of data from the target domain. Thus, we propose a novel method, called Domain Adaptive Distill-Tuning (DADT), to adapt a pre-trained model with limited target data (approximately 100 LiDAR frames), retaining its representation power and preventing it from overfitting. Specifically, we use regularizers to align object-level and context-level representations between the pre-trained and finetuned models in a teacher-student architecture. Our experiments with driving benchmarks, i.e., Waymo Open dataset and KITTI, confirm that our method effectively finetunes a pre-trained model, achieving significant gains in accuracy.