PointCubeNet: 3D Part-level Reasoning with 3x3x3 Point Cloud Blocks

TL;DR

PointCubeNet introduces an unsupervised framework for 3D part-level reasoning using 3x3x3 point cloud blocks, improving 3D object understanding without requiring part annotations.

Contribution

It is the first to perform unsupervised 3D part-level reasoning with a novel multi-modal framework and local block structure.

Findings

Enhances 3D object understanding through part-level analysis.

Effective unsupervised training via pseudo-labeling and local loss.

Achieves reliable 3D part reasoning without annotations.

Abstract

In this paper, we propose PointCubeNet, a novel multi-modal 3D understanding framework that achieves part-level reasoning without requiring any part annotations. PointCubeNet comprises global and local branches. The proposed local branch, structured into 3x3x3 local blocks, enables part-level analysis of point cloud sub-regions with the corresponding local text labels. Leveraging the proposed pseudo-labeling method and local loss function, PointCubeNet is effectively trained in an unsupervised manner. The experimental results demonstrate that understanding 3D object parts enhances the understanding of the overall 3D object. In addition, this is the first attempt to perform unsupervised 3D part-level reasoning and achieves reliable and meaningful results.