# Learning Physics-guided Face Relighting under Directional Light

## TL;DR

This paper presents a deep learning approach for face relighting that decomposes images into intrinsic components and models non-diffuse effects, enabling realistic relighting in complex lighting and pose scenarios.

## Contribution

The authors propose an end-to-end neural network that combines physics-based decomposition with residual correction for non-diffuse effects, trained on a new portrait dataset with diverse lighting and poses.

## Key findings

- Achieves precise and believable face relighting results.
- Generalizes well to complex illumination and challenging poses.
- Outperforms existing methods in relighting accuracy.

## Abstract

Relighting is an essential step in realistically transferring objects from a captured image into another environment. For example, authentic telepresence in Augmented Reality requires faces to be displayed and relit consistent with the observer's scene lighting. We investigate end-to-end deep learning architectures that both de-light and relight an image of a human face. Our model decomposes the input image into intrinsic components according to a diffuse physics-based image formation model. We enable non-diffuse effects including cast shadows and specular highlights by predicting a residual correction to the diffuse render. To train and evaluate our model, we collected a portrait database of 21 subjects with various expressions and poses. Each sample is captured in a controlled light stage setup with 32 individual light sources. Our method creates precise and believable relighting results and generalizes to complex illumination conditions and challenging poses, including when the subject is not looking straight at the camera.