Deep, Dense, and Low-Rank Gaussian Conditional Random Fields

TL;DR

This paper introduces a novel deep Gaussian CRF model with fully-connected graphs, low-rank embeddings, and efficient training, achieving state-of-the-art results in several computer vision tasks.

Contribution

It presents a new low-rank, fully-connected Gaussian CRF with end-to-end training and efficient inference for improved computer vision performance.

Findings

Achieved state-of-the-art results on semantic segmentation, human parts segmentation, and saliency estimation.

Developed an end-to-end trainable deep Gaussian CRF with low-rank embeddings.

Implemented a GPU-based solution for efficient computation.

Abstract

In this work we introduce a fully-connected graph structure in the Deep Gaussian Conditional Random Field (G-CRF) model. For this we express the pairwise interactions between pixels as the inner-products of low-dimensional embeddings, delivered by a new subnetwork of a deep architecture. We efficiently minimize the resulting energy by solving the resulting low-rank linear system with conjugate gradients, and derive an analytic expression for the gradient of our embeddings which allows us to train them end-to-end with backpropagation. We demonstrate the merit of our approach by achieving state of the art results on three challenging Computer Vision benchmarks, namely semantic segmentation, human parts segmentation, and saliency estimation. Our implementation is fully GPU based, built on top of the Caffe library, and will be made publicly available.