Discovering 3D Parts from Image Collections

TL;DR

This paper introduces a self-supervised method for discovering 3D object parts from collections of 2D images, enabling shape reconstruction without manual part annotations.

Contribution

It proposes a novel latent part discovery approach that learns a shape prior for parts, improving 3D shape understanding from 2D images without supervision.

Findings

Discovered consistent object parts across datasets

Achieved competitive shape reconstruction accuracy

Validated on synthetic and real-world datasets

Abstract

Reasoning 3D shapes from 2D images is an essential yet challenging task, especially when only single-view images are at our disposal. While an object can have a complicated shape, individual parts are usually close to geometric primitives and thus are easier to model. Furthermore, parts provide a mid-level representation that is robust to appearance variations across objects in a particular category. In this work, we tackle the problem of 3D part discovery from only 2D image collections. Instead of relying on manually annotated parts for supervision, we propose a self-supervised approach, latent part discovery (LPD). Our key insight is to learn a novel part shape prior that allows each part to fit an object shape faithfully while constrained to have simple geometry. Extensive experiments on the synthetic ShapeNet, PartNet, and real-world Pascal 3D+ datasets show that our method discovers consistent object parts and achieves favorable reconstruction accuracy compared to the existing methods with the same level of supervision.