Geometry-Aware Network for Non-Rigid Shape Prediction from a Single View

TL;DR

This paper introduces a geometry-aware deep learning method for predicting 3D shapes of deformable surfaces from a single image, without needing pre-registered templates, and demonstrates improved accuracy and efficiency over existing methods.

Contribution

The proposed approach is the first to combine geometry-awareness with deep learning for single-view non-rigid shape prediction without template registration.

Findings

Outperforms state-of-the-art methods in accuracy.

Reduces computational time significantly.

Effective on both synthetic and real datasets.

Abstract

We propose a method for predicting the 3D shape of a deformable surface from a single view. By contrast with previous approaches, we do not need a pre-registered template of the surface, and our method is robust to the lack of texture and partial occlusions. At the core of our approach is a {\it geometry-aware} deep architecture that tackles the problem as usually done in analytic solutions: first perform 2D detection of the mesh and then estimate a 3D shape that is geometrically consistent with the image. We train this architecture in an end-to-end manner using a large dataset of synthetic renderings of shapes under different levels of deformation, material properties, textures and lighting conditions. We evaluate our approach on a test split of this dataset and available real benchmarks, consistently improving state-of-the-art solutions with a significantly lower computational time.