Adaptive 3D Face Reconstruction from a Single Image

TL;DR

This paper introduces a joint 2D-3D optimization approach for single-image 3D face reconstruction, effectively handling occlusions and extreme poses by adaptively refining pose and landmark weights.

Contribution

It presents a novel adaptive method combining 2D and 3D data for improved accuracy in challenging conditions, outperforming existing techniques.

Findings

High-quality 3D face reconstructions from single images

Robustness to occlusions and large pose variations

Effective adaptive weighting strategy

Abstract

3D face reconstruction from a single image is a challenging problem, especially under partial occlusions and extreme poses. This is because the uncertainty of the estimated 2D landmarks will affect the quality of face reconstruction. In this paper, we propose a novel joint 2D and 3D optimization method to adaptively reconstruct 3D face shapes from a single image, which combines the depths of 3D landmarks to solve the uncertain detections of invisible landmarks. The strategy of our method involves two aspects: a coarse-to-fine pose estimation using both 2D and 3D landmarks, and an adaptive 2D and 3D re-weighting based on the refined pose parameter to recover accurate 3D faces. Experimental results on multiple datasets demonstrate that our method can generate high-quality reconstruction from a single color image and is robust for self-occlusion and large poses.