MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views

TL;DR

MHR-Net introduces a novel unsupervised approach for reconstructing non-rigid 3D shapes from 2D views by generating multiple hypotheses and selecting the most accurate one, achieving state-of-the-art results.

Contribution

The paper presents a new deterministic and stochastic framework for non-rigid shape reconstruction, including a hypothesis generation scheme and a Procrustean Residual Loss for improved accuracy.

Findings

Achieves state-of-the-art accuracy on Human3.6M, SURREAL, and 300-VW datasets.

Effectively models uncertainty in non-rigid shape deformation.

Improves reconstruction quality with the Procrustean Residual Loss.

Abstract

We propose MHR-Net, a novel method for recovering Non-Rigid Shapes from Motion (NRSfM). MHR-Net aims to find a set of reasonable reconstructions for a 2D view, and it also selects the most likely reconstruction from the set. To deal with the challenging unsupervised generation of non-rigid shapes, we develop a new Deterministic Basis and Stochastic Deformation scheme in MHR-Net. The non-rigid shape is first expressed as the sum of a coarse shape basis and a flexible shape deformation, then multiple hypotheses are generated with uncertainty modeling of the deformation part. MHR-Net is optimized with reprojection loss on the basis and the best hypothesis. Furthermore, we design a new Procrustean Residual Loss, which reduces the rigid rotations between similar shapes and further improves the performance. Experiments show that MHR-Net achieves state-of-the-art reconstruction accuracy on Human3.6M, SURREAL and 300-VW datasets.