S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Image

TL;DR

This paper introduces S2F2, a self-supervised deep learning method for high-fidelity, real-time face reconstruction from a single image, capturing detailed geometry and reflectance with improved visual quality.

Contribution

It presents the first self-supervised approach achieving high-fidelity face reconstruction with detailed geometry and reflectance from monocular images.

Findings

Outperforms existing methods in visual quality.

Achieves real-time reconstruction speeds.

Successfully decouples face reflectance from geometry.

Abstract

We present a novel face reconstruction method capable of reconstructing detailed face geometry, spatially varying face reflectance from a single monocular image. We build our work upon the recent advances of DNN-based auto-encoders with differentiable ray tracing image formation, trained in self-supervised manner. While providing the advantage of learning-based approaches and real-time reconstruction, the latter methods lacked fidelity. In this work, we achieve, for the first time, high fidelity face reconstruction using self-supervised learning only. Our novel coarse-to-fine deep architecture allows us to solve the challenging problem of decoupling face reflectance from geometry using a single image, at high computational speed. Compared to state-of-the-art methods, our method achieves more visually appealing reconstruction.