BioFaceNet: Deep Biophysical Face Image Interpretation

TL;DR

BioFaceNet is a deep learning model that interprets face images by decomposing them into biophysical parameters, shading maps, and spectral information, enabling detailed analysis of facial appearance in unconstrained settings.

Contribution

The paper introduces BioFaceNet, a novel deep CNN that performs comprehensive biophysical decomposition of face images using self-supervised learning and model-based priors, advancing face image interpretation.

Findings

Qualitative results show effective decomposition on in-the-wild data.

Introduces a new benchmark for quantitative evaluation of face decomposition.

Demonstrates the model's ability to estimate spectral and biophysical parameters accurately.

Abstract

In this paper we present BioFaceNet, a deep CNN that learns to decompose a single face image into biophysical parameters maps, diffuse and specular shading maps as well as estimating the spectral power distribution of the scene illuminant and the spectral sensitivity of the camera. The network comprises a fully convolutional encoder for estimating the spatial maps with a fully connected branch for estimating the vector quantities. The network is trained using a self-supervised appearance loss computed via a model-based decoder. The task is highly underconstrained so we impose a number of model-based priors. Skin spectral reflectance is restricted to a biophysical model, we impose a statistical prior on camera spectral sensitivities, a physical constraint on illumination spectra, a sparsity prior on specular reflections and direct supervision on diffuse shading using a rough shape proxy. We show convincing qualitative results on in-the-wild data and introduce a benchmark for quantitative evaluation on this new task.