Generative Data Augmentation for Object Point Cloud Segmentation

TL;DR

This paper introduces a novel 3D diffusion-based generative data augmentation method for point cloud segmentation, significantly improving model performance with limited labeled data by generating high-quality, semantically labeled point clouds.

Contribution

It extends the Lion diffusion model to a part-aware generator conditioned on segmentation masks and proposes a 3-step GDA pipeline with pseudo-label filtering for enhanced segmentation training.

Findings

Outperforms traditional augmentation methods in segmentation accuracy.

Effective with limited labeled data, reducing annotation costs.

Validated on synthetic and real-world datasets.

Abstract

Data augmentation is widely used to train deep learning models to address data scarcity. However, traditional data augmentation (TDA) typically relies on simple geometric transformation, such as random rotation and rescaling, resulting in minimal data diversity enrichment and limited model performance improvement. State-of-the-art generative models for 3D shape generation rely on the denoising diffusion probabilistic models and manage to generate realistic novel point clouds for 3D content creation and manipulation. Nevertheless, the generated 3D shapes lack associated point-wise semantic labels, restricting their usage in enlarging the training data for point cloud segmentation tasks. To bridge the gap between data augmentation techniques and the advanced diffusion models, we extend the state-of-the-art 3D diffusion model, Lion, to a part-aware generative model that can generate high-quality point clouds conditioned on given segmentation masks. Leveraging the novel generative model, we introduce a 3-step generative data augmentation (GDA) pipeline for point cloud segmentation training. Our GDA approach requires only a small amount of labeled samples but enriches the training data with generated variants and pseudo-labeled samples, which are validated by a novel diffusion-based pseudo-label filtering method. Extensive experiments on two large-scale synthetic datasets and a real-world medical dataset demonstrate that our GDA method outperforms TDA approach and related semi-supervised and self-supervised methods.