# Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation

## TL;DR

This paper introduces two crossmodal knowledge distillation methods that leverage synchronized 2D camera images to improve 3D LiDAR semantic segmentation without requiring extensive 3D annotations, using domain adaptation and feature alignment techniques.

## Contribution

It proposes novel unsupervised domain adaptation and feature distillation methods that utilize 2D-3D data correspondence to reduce annotation costs in 3D semantic segmentation.

## Key findings

- Outperforms state-of-the-art methods in 3D segmentation accuracy.
- Effectively leverages synchronized 2D images for 3D model training.
- Demonstrates robustness across different autonomous driving datasets.

## Abstract

Semantic segmentation of 3D LiDAR data plays a pivotal role in autonomous driving. Traditional approaches rely on extensive annotated data for point cloud analysis, incurring high costs and time investments. In contrast, realworld image datasets offer abundant availability and substantial scale. To mitigate the burden of annotating 3D LiDAR point clouds, we propose two crossmodal knowledge distillation methods: Unsupervised Domain Adaptation Knowledge Distillation (UDAKD) and Feature and Semantic-based Knowledge Distillation (FSKD). Leveraging readily available spatio-temporally synchronized data from cameras and LiDARs in autonomous driving scenarios, we directly apply a pretrained 2D image model to unlabeled 2D data. Through crossmodal knowledge distillation with known 2D-3D correspondence, we actively align the output of the 3D network with the corresponding points of the 2D network, thereby obviating the necessity for 3D annotations. Our focus is on preserving modality-general information while filtering out modality-specific details during crossmodal distillation. To achieve this, we deploy self-calibrated convolution on 3D point clouds as the foundation of our domain adaptation module. Rigorous experimentation validates the effectiveness of our proposed methods, consistently surpassing the performance of state-of-the-art approaches in the field.