Face Parsing via a Fully-Convolutional Continuous CRF Neural Network

TL;DR

This paper introduces a fully convolutional neural network architecture with integrated continuous CRF for face parsing, achieving high accuracy and spatial consistency without region-based subnetworks.

Contribution

The novel FC-CNN framework combines unary, pairwise, and continuous CRF subnetworks into a unified, fully convolutional model for improved face parsing accuracy.

Findings

Outperforms state-of-the-art methods on LFW-PL and HELEN datasets.

Achieves high segmentation accuracy with spatial consistency.

Efficiently captures semantic and edge information across layers.

Abstract

In this work, we address the face parsing task with a Fully-Convolutional continuous CRF Neural Network (FC-CNN) architecture. In contrast to previous face parsing methods that apply region-based subnetwork hundreds of times, our FC-CNN is fully convolutional with high segmentation accuracy. To achieve this goal, FC-CNN integrates three subnetworks, a unary network, a pairwise network and a continuous Conditional Random Field (C-CRF) network into a unified framework. The high-level semantic information and low-level details across different convolutional layers are captured by the convolutional and deconvolutional structures in the unary network. The semantic edge context is learnt by the pairwise network branch to construct pixel-wise affinity. Based on a differentiable superpixel pooling layer and a differentiable C-CRF layer, the unary network and pairwise network are combined via a novel continuous CRF network to achieve spatial consistency in both training and test procedure of a deep neural network. Comprehensive evaluations on LFW-PL and HELEN datasets demonstrate that FC-CNN achieves better performance over the other state-of-arts for accurate face labeling on challenging images.