SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery

TL;DR

SIDER is a novel unsupervised method that recovers detailed 3D facial geometry from a single image using neural optimization and implicit representations, without requiring datasets or multi-view supervision.

Contribution

It introduces a single-image neural optimization approach combining statistical shape priors and implicit neural representations for detailed facial geometry recovery.

Findings

Achieves state-of-the-art detail recovery from a single image.

Does not require dataset priors or multi-view supervision.

Performs well on in-the-wild images.

Abstract

We present SIDER(Single-Image neural optimization for facial geometric DEtail Recovery), a novel photometric optimization method that recovers detailed facial geometry from a single image in an unsupervised manner. Inspired by classical techniques of coarse-to-fine optimization and recent advances in implicit neural representations of 3D shape, SIDER combines a geometry prior based on statistical models and Signed Distance Functions (SDFs) to recover facial details from single images. First, it estimates a coarse geometry using a morphable model represented as an SDF. Next, it reconstructs facial geometry details by optimizing a photometric loss with respect to the ground truth image. In contrast to prior work, SIDER does not rely on any dataset priors and does not require additional supervision from multiple views, lighting changes or ground truth 3D shape. Extensive qualitative and quantitative evaluation demonstrates that our method achieves state-of-the-art on facial geometric detail recovery, using only a single in-the-wild image.