Disjoint Pose and Shape for 3D Face Reconstruction

TL;DR

This paper introduces an end-to-end method for 3D face reconstruction from limited images by separately estimating pose and shape, leading to more stable and accurate results compared to existing approaches.

Contribution

It proposes a novel disjoint approach to jointly optimize face pose and shape, improving stability and accuracy over prior methods that combine deep learning and multi-view stereo techniques.

Findings

Achieves end-to-end topological consistency.

Enables iterative face pose refinement.

Shows significant improvements over state-of-the-art methods.

Abstract

Existing methods for 3D face reconstruction from a few casually captured images employ deep learning based models along with a 3D Morphable Model(3DMM) as face geometry prior. Structure From Motion(SFM), followed by Multi-View Stereo (MVS), on the other hand, uses dozens of high-resolution images to reconstruct accurate 3D faces.However, it produces noisy and stretched-out results with only two views available. In this paper, taking inspiration from both these methods, we propose an end-to-end pipeline that disjointly solves for pose and shape to make the optimization stable and accurate. We use a face shape prior to estimate face pose and use stereo matching followed by a 3DMM to solve for the shape. The proposed method achieves end-to-end topological consistency, enables iterative face pose refinement procedure, and show remarkable improvement on both quantitative and qualitative results over existing state-of-the-art methods.