Quasi-Balanced Self-Training on Noise-Aware Synthesis of Object Point Clouds for Closing Domain Gap

TL;DR

This paper introduces a novel approach combining physically realistic synthetic point cloud generation with a quasi-balanced self-training method to improve domain adaptation for object classification in noisy, incomplete real-world point clouds.

Contribution

It proposes a new synthetic data generation method using speckle pattern rendering and a quasi-balanced self-training algorithm for better domain adaptation in point cloud classification.

Findings

Achieved state-of-the-art results on unsupervised domain adaptation tasks.

Demonstrated the effectiveness of physically realistic synthetic data.

Validated the proposed quasi-balanced self-training approach.

Abstract

Semantic analyses of object point clouds are largely driven by releasing of benchmarking datasets, including synthetic ones whose instances are sampled from object CAD models. However, learning from synthetic data may not generalize to practical scenarios, where point clouds are typically incomplete, non-uniformly distributed, and noisy. Such a challenge of Simulation-to-Reality (Sim2Real) domain gap could be mitigated via learning algorithms of domain adaptation; however, we argue that generation of synthetic point clouds via more physically realistic rendering is a powerful alternative, as systematic non-uniform noise patterns can be captured. To this end, we propose an integrated scheme consisting of physically realistic synthesis of object point clouds via rendering stereo images via projection of speckle patterns onto CAD models and a novel quasi-balanced self-training designed for more balanced data distribution by sparsity-driven selection of pseudo labeled samples for long tailed classes. Experiment results can verify the effectiveness of our method as well as both of its modules for unsupervised domain adaptation on point cloud classification, achieving the state-of-the-art performance. Source codes and the SpeckleNet synthetic dataset are available at https://github.com/Gorilla-Lab-SCUT/QS3.