Efficient Linear Programming for Dense CRFs

TL;DR

This paper introduces a fast and efficient linear programming algorithm for dense CRFs used in semantic segmentation, significantly improving speed while maintaining accuracy.

Contribution

The authors develop a novel proximal minimization framework with block coordinate descent, enabling linear-time conditional gradient computations for dense CRFs.

Findings

Outperforms existing dense CRF algorithms in speed and accuracy

Reduces LP minimization to pixel-wise subproblems

Achieves state-of-the-art results on standard datasets

Abstract

The fully connected conditional random field (CRF) with Gaussian pairwise potentials has proven popular and effective for multi-class semantic segmentation. While the energy of a dense CRF can be minimized accurately using a linear programming (LP) relaxation, the state-of-the-art algorithm is too slow to be useful in practice. To alleviate this deficiency, we introduce an efficient LP minimization algorithm for dense CRFs. To this end, we develop a proximal minimization framework, where the dual of each proximal problem is optimized via block coordinate descent. We show that each block of variables can be efficiently optimized. Specifically, for one block, the problem decomposes into significantly smaller subproblems, each of which is defined over a single pixel. For the other block, the problem is optimized via conditional gradient descent. This has two advantages: 1) the conditional gradient can be computed in a time linear in the number of pixels and labels; and 2) the optimal step size can be computed analytically. Our experiments on standard datasets provide compelling evidence that our approach outperforms all existing baselines including the previous LP based approach for dense CRFs.