Semantic-Free Procedural 3D Shapes Are Surprisingly Good Teachers

TL;DR

This paper demonstrates that self-supervised learning from procedurally generated 3D shapes without semantic content can achieve performance comparable to models trained on semantically labeled data across various 3D tasks.

Contribution

It introduces a novel approach of using semantic-free procedural 3D programs for self-supervised learning, challenging the reliance on semantic information in 3D representation learning.

Findings

Procedurally generated shapes perform on par with semantically labeled models

Current 3D self-supervised methods do not depend on shape semantics

Procedural programs can effectively generate training data for 3D tasks

Abstract

Self-supervised learning has emerged as a promising approach for acquiring transferable 3D representations from unlabeled 3D point clouds. Unlike 2D images, which are widely accessible, acquiring 3D assets requires specialized expertise or professional 3D scanning equipment, making it difficult to scale and raising copyright concerns. To address these challenges, we propose learning 3D representations from procedural 3D programs that automatically generate 3D shapes using simple 3D primitives and augmentations. Remarkably, despite lacking semantic content, the 3D representations learned from the procedurally generated 3D shapes perform on par with state-of-the-art representations learned from semantically recognizable 3D models (e.g., airplanes) across various downstream 3D tasks, such as shape classification, part segmentation, masked point cloud completion, and both scene semantic and instance segmentation. We provide a detailed analysis on factors that make a good 3D procedural programs. Extensive experiments further suggest that current 3D self-supervised learning methods on point clouds do not rely on semantics of 3D shapes, shedding light on the nature of 3D representations learned.