Learning to Reconstruct Shapes from Unseen Classes

TL;DR

This paper introduces GenRe, a novel algorithm for 3D shape reconstruction from a single image that generalizes well to unseen object categories by leveraging class-agnostic shape priors and a combination of 2.5D, spherical, and voxel representations.

Contribution

The paper presents GenRe, a new inference network and training method that captures generic shape priors, enabling better generalization in single-image 3D reconstruction tasks.

Findings

GenRe outperforms existing methods on single-view shape reconstruction.

It generalizes effectively to novel object categories not seen during training.

The approach combines multiple shape representations to improve accuracy.

Abstract

From a single image, humans are able to perceive the full 3D shape of an object by exploiting learned shape priors from everyday life. Contemporary single-image 3D reconstruction algorithms aim to solve this task in a similar fashion, but often end up with priors that are highly biased by training classes. Here we present an algorithm, Generalizable Reconstruction (GenRe), designed to capture more generic, class-agnostic shape priors. We achieve this with an inference network and training procedure that combine 2.5D representations of visible surfaces (depth and silhouette), spherical shape representations of both visible and non-visible surfaces, and 3D voxel-based representations, in a principled manner that exploits the causal structure of how 3D shapes give rise to 2D images. Experiments demonstrate that GenRe performs well on single-view shape reconstruction, and generalizes to diverse novel objects from categories not seen during training.