DODA: Data-oriented Sim-to-Real Domain Adaptation for 3D Semantic Segmentation

TL;DR

This paper introduces DODA, a novel data-oriented domain adaptation framework that improves 3D semantic segmentation from synthetic to real data by simulating real-world patterns and reducing interior context gaps, achieving state-of-the-art results.

Contribution

The paper proposes a new domain adaptation method combining virtual scan simulation and tail-aware cuboid mixing for better sim-to-real transfer in 3D segmentation.

Findings

DODA surpasses existing UDA methods by over 13% in accuracy.

Built the first unsupervised sim-to-real benchmark for 3D indoor segmentation.

Achieved significant domain gap reduction in 3D semantic segmentation tasks.

Abstract

Deep learning approaches achieve prominent success in 3D semantic segmentation. However, collecting densely annotated real-world 3D datasets is extremely time-consuming and expensive. Training models on synthetic data and generalizing on real-world scenarios becomes an appealing alternative, but unfortunately suffers from notorious domain shifts. In this work, we propose a Data-Oriented Domain Adaptation (DODA) framework to mitigate pattern and context gaps caused by different sensing mechanisms and layout placements across domains. Our DODA encompasses virtual scan simulation to imitate real-world point cloud patterns and tail-aware cuboid mixing to alleviate the interior context gap with a cuboid-based intermediate domain. The first unsupervised sim-to-real adaptation benchmark on 3D indoor semantic segmentation is also built on 3D-FRONT, ScanNet and S3DIS along with 7 popular Unsupervised Domain Adaptation (UDA) methods. Our DODA surpasses existing UDA approaches by over 13% on both 3D-FRONT -> ScanNet and 3D-FRONT -> S3DIS. Code is available at https://github.com/CVMI-Lab/DODA.