Robust Deep Appearance Models

TL;DR

This paper introduces Robust Deep Appearance Models that effectively learn shape and texture correlations in face images, especially handling occlusions and corruptions for improved face reconstruction and fitting.

Contribution

It proposes a novel combination of Deep Boltzmann Machines and Robust Deep Boltzmann Machines for joint shape and texture modeling with occlusion robustness.

Findings

Enhanced face reconstruction accuracy on challenging datasets

Effective occlusion handling through mask-based fitting algorithms

Robustness demonstrated across multiple face datasets

Abstract

This paper presents a novel Robust Deep Appearance Models to learn the non-linear correlation between shape and texture of face images. In this approach, two crucial components of face images, i.e. shape and texture, are represented by Deep Boltzmann Machines and Robust Deep Boltzmann Machines (RDBM), respectively. The RDBM, an alternative form of Robust Boltzmann Machines, can separate corrupted/occluded pixels in the texture modeling to achieve better reconstruction results. The two models are connected by Restricted Boltzmann Machines at the top layer to jointly learn and capture the variations of both facial shapes and appearances. This paper also introduces new fitting algorithms with occlusion awareness through the mask obtained from the RDBM reconstruction. The proposed approach is evaluated in various applications by using challenging face datasets, i.e. Labeled Face Parts in the Wild (LFPW), Helen, EURECOM and AR databases, to demonstrate its robustness and capabilities.