Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding

TL;DR

This paper introduces a novel generative Bayesian network for creating diverse synthetic 3D scenes to improve self-supervised representation learning, enhancing transferability to real-world 3D scene understanding tasks.

Contribution

It proposes a new synthetic data generation method combined with joint contrastive and occlusion-aware reconstruction learning for better 3D scene representations.

Findings

Outperforms existing pre-training methods on 3D detection tasks

Achieves superior results on 3D semantic segmentation benchmarks

Demonstrates effective transfer of synthetic pre-training to real-world data

Abstract

The field of self-supervised 3D representation learning has emerged as a promising solution to alleviate the challenge presented by the scarcity of extensive, well-annotated datasets. However, it continues to be hindered by the lack of diverse, large-scale, real-world 3D scene datasets for source data. To address this shortfall, we propose Generalizable Representation Learning (GRL), where we devise a generative Bayesian network to produce diverse synthetic scenes with real-world patterns, and conduct pre-training with a joint objective. By jointly learning a coarse-to-fine contrastive learning task and an occlusion-aware reconstruction task, the model is primed with transferable, geometry-informed representations. Post pre-training on synthetic data, the acquired knowledge of the model can be seamlessly transferred to two principal downstream tasks associated with 3D scene understanding, namely 3D object detection and 3D semantic segmentation, using real-world benchmark datasets. A thorough series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.