Category-Specific Object Reconstruction from a Single Image

TL;DR

This paper presents an automated pipeline for reconstructing 3D object surfaces from single images across various categories, leveraging deformable models learned from 2D annotations and detailed shape recovery techniques.

Contribution

It introduces a novel method that combines deformable 3D models learned from 2D data with a bottom-up detail recovery module for single-image object reconstruction.

Findings

Effective reconstruction on PASCAL 3D+ dataset

Robustness to noisy segmentations demonstrated

Significant improvement over baseline methods

Abstract

Object reconstruction from a single image -- in the wild -- is a problem where we can make progress and get meaningful results today. This is the main message of this paper, which introduces an automated pipeline with pixels as inputs and 3D surfaces of various rigid categories as outputs in images of realistic scenes. At the core of our approach are deformable 3D models that can be learned from 2D annotations available in existing object detection datasets, that can be driven by noisy automatic object segmentations and which we complement with a bottom-up module for recovering high-frequency shape details. We perform a comprehensive quantitative analysis and ablation study of our approach using the recently introduced PASCAL 3D+ dataset and show very encouraging automatic reconstructions on PASCAL VOC.