Syn-to-Real Unsupervised Domain Adaptation for Indoor 3D Object Detection

TL;DR

This paper introduces a novel unsupervised domain adaptation framework for indoor 3D object detection, effectively bridging the gap between synthetic and real datasets through hierarchical alignment and pseudo labeling.

Contribution

It proposes the Object-wise Hierarchical Domain Alignment (OHDA) framework with object-aware augmentation and dual-branch adaptation for improved syn-to-real indoor 3D detection.

Findings

Achieves 9.7% and 9.1% mAP improvements on ScanNetV2 and SUN RGB-D.

Outperforms existing outdoor scenario adaptation methods.

Demonstrates effective domain alignment across synthetic and real indoor datasets.

Abstract

The use of synthetic data in indoor 3D object detection offers the potential of greatly reducing the manual labor involved in 3D annotations and training effective zero-shot detectors. However, the complicated domain shifts across syn-to-real indoor datasets remains underexplored. In this paper, we propose a novel Object-wise Hierarchical Domain Alignment (OHDA) framework for syn-to-real unsupervised domain adaptation in indoor 3D object detection. Our approach includes an object-aware augmentation strategy to effectively diversify the source domain data, and we introduce a two-branch adaptation framework consisting of an adversarial training branch and a pseudo labeling branch, in order to simultaneously reach holistic-level and class-level domain alignment. The pseudo labeling is further refined through two proposed schemes specifically designed for indoor UDA. Our adaptation results from synthetic dataset 3D-FRONT to real-world datasets ScanNetV2 and SUN RGB-D demonstrate remarkable mAP25 improvements of 9.7% and 9.1% over Source-Only baselines, respectively, and consistently outperform the methods adapted from 2D and 3D outdoor scenarios. The code will be publicly available upon paper acceptance.