Planes vs. Chairs: Category-guided 3D shape learning without any 3D cues

TL;DR

This paper introduces a category-guided 3D shape learning method that reconstructs 3D shapes from single RGB images across multiple categories without 3D supervision, using novel metric learning and regularization techniques.

Contribution

It is the first to perform large-scale single-view 3D shape prediction across over 50 categories without 3D cues, leveraging category labels for improved learning.

Findings

Achieves state-of-the-art results on ShapeNet-13, ShapeNet-55, and Pascal3D+ datasets.

Demonstrates the benefit of class information in 3D shape reconstruction.

First to quantify the impact of category labels in single-view 3D shape learning.

Abstract

We present a novel 3D shape reconstruction method which learns to predict an implicit 3D shape representation from a single RGB image. Our approach uses a set of single-view images of multiple object categories without viewpoint annotation, forcing the model to learn across multiple object categories without 3D supervision. To facilitate learning with such minimal supervision, we use category labels to guide shape learning with a novel categorical metric learning approach. We also utilize adversarial and viewpoint regularization techniques to further disentangle the effects of viewpoint and shape. We obtain the first results for large-scale (more than 50 categories) single-viewpoint shape prediction using a single model without any 3D cues. We are also the first to examine and quantify the benefit of class information in single-view supervised 3D shape reconstruction. Our method achieves superior performance over state-of-the-art methods on ShapeNet-13, ShapeNet-55 and Pascal3D+.