Convolutional CRFs for Semantic Segmentation

TL;DR

This paper introduces convolutional CRFs that reformulate inference as convolutions, significantly speeding up training and inference for semantic segmentation, and enabling end-to-end learning of CRF parameters.

Contribution

It proposes a novel convolutional CRF model that allows efficient GPU implementation and end-to-end training, addressing previous speed and learnability limitations.

Findings

Inference and training are sped up by over 100 times.

All CRF parameters can be optimized via backpropagation.

The implementation is publicly available for further research.

Abstract

For the challenging semantic image segmentation task the most efficient models have traditionally combined the structured modelling capabilities of Conditional Random Fields (CRFs) with the feature extraction power of CNNs. In more recent works however, CRF post-processing has fallen out of favour. We argue that this is mainly due to the slow training and inference speeds of CRFs, as well as the difficulty of learning the internal CRF parameters. To overcome both issues we propose to add the assumption of conditional independence to the framework of fully-connected CRFs. This allows us to reformulate the inference in terms of convolutions, which can be implemented highly efficiently on GPUs. Doing so speeds up inference and training by a factor of more then 100. All parameters of the convolutional CRFs can easily be optimized using backpropagation. To facilitating further CRF research we make our implementation publicly available. Please visit: https://github.com/MarvinTeichmann/ConvCRF