DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

TL;DR

DenseReg introduces a fully convolutional network that learns dense pixel-to-template mappings for faces and bodies, improving landmark localization and dense correspondence estimation in unconstrained images.

Contribution

It presents a novel dense regression architecture combining segmentation ideas with regression, enabling accurate dense correspondence in-the-wild.

Findings

Outperforms state-of-the-art on 300W landmark localization

Provides dense human body correspondence results

Offers a fully convolutional, stand-alone dense registration system

Abstract

In this paper we propose to learn a mapping from image pixels into a dense template grid through a fully convolutional network. We formulate this task as a regression problem and train our network by leveraging upon manually annotated facial landmarks "in-the-wild". We use such landmarks to establish a dense correspondence field between a three-dimensional object template and the input image, which then serves as the ground-truth for training our regression system. We show that we can combine ideas from semantic segmentation with regression networks, yielding a highly-accurate "quantized regression" architecture. Our system, called DenseReg, allows us to estimate dense image-to-template correspondences in a fully convolutional manner. As such our network can provide useful correspondence information as a stand-alone system, while when used as an initialization for Statistical Deformable Models we obtain landmark localization results that largely outperform the current state-of-the-art on the challenging 300W benchmark. We thoroughly evaluate our method on a host of facial analysis tasks and also provide qualitative results for dense human body correspondence. We make our code available at http://alpguler.com/DenseReg.html along with supplementary materials.