PAUL: Procrustean Autoencoder for Unsupervised Lifting

TL;DR

PAUL introduces a novel Procrustean autoencoder framework that learns 3D shape representations from 2D measurements alone, jointly estimating pose and shape for improved unsupervised 3D reconstruction.

Contribution

It presents a unique autoencoder architecture that learns from 2D data and jointly estimates pose and shape, advancing unsupervised NRSfM methods.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Effectively learns 3D shapes solely from 2D projections.

Joint pose and shape estimation improves reconstruction accuracy.

Abstract

Recent success in casting Non-rigid Structure from Motion (NRSfM) as an unsupervised deep learning problem has raised fundamental questions about what novelty in NRSfM prior could the deep learning offer. In this paper we advocate for a 3D deep auto-encoder framework to be used explicitly as the NRSfM prior. The framework is unique as: (i) it learns the 3D auto-encoder weights solely from 2D projected measurements, and (ii) it is Procrustean in that it jointly resolves the unknown rigid pose for each shape instance. We refer to this architecture as a Procustean Autoencoder for Unsupervised Lifting (PAUL), and demonstrate state-of-the-art performance across a number of benchmarks in comparison to recent innovations such as Deep NRSfM and C3PDO.