Face Inverse Rendering via Hierarchical Decoupling

TL;DR

This paper introduces a hierarchical deep learning framework for face inverse rendering from in-the-wild images, effectively disentangling albedo, normals, and lighting without relying on synthetic data or professional equipment.

Contribution

The proposed method employs a hierarchical subdivision strategy to enable face inverse rendering using arbitrary viewpoint image pairs, reducing data preparation requirements and enhancing real-world applicability.

Findings

Demonstrates superior face relighting performance compared to state-of-the-art methods.

Effectively disentangles face components from in-the-wild images.

Broadens the applicability of face inverse rendering without synthetic data.

Abstract

Previous face inverse rendering methods often require synthetic data with ground truth and/or professional equipment like a lighting stage. However, a model trained on synthetic data or using pre-defined lighting priors is typically unable to generalize well for real-world situations, due to the gap between synthetic data/lighting priors and real data. Furthermore, for common users, the professional equipment and skill make the task expensive and complex. In this paper, we propose a deep learning framework to disentangle face images in the wild into their corresponding albedo, normal, and lighting components. Specifically, a decomposition network is built with a hierarchical subdivision strategy, which takes image pairs captured from arbitrary viewpoints as input. In this way, our approach can greatly mitigate the pressure from data preparation, and significantly broaden the applicability of face inverse rendering. Extensive experiments are conducted to demonstrate the efficacy of our design, and show its superior performance in face relighting over other state-of-the-art alternatives. {Our code is available at \url{https://github.com/AutoHDR/HD-Net.git}}