Efficient Continuous Relaxations for Dense CRF

TL;DR

This paper introduces efficient continuous relaxation methods for dense CRFs, improving optimization speed and theoretical guarantees over traditional mean-field algorithms in computer vision tasks.

Contribution

It presents novel convex and difference-of-convex relaxation techniques, along with a divide-and-conquer subgradient computation, enhancing energy minimization in dense CRFs.

Findings

Continuous relaxations outperform mean-field in accuracy.

Proposed methods offer faster convergence and better theoretical bounds.

Demonstrated improvements on benchmark datasets.

Abstract

Dense conditional random fields (CRF) with Gaussian pairwise potentials have emerged as a popular framework for several computer vision applications such as stereo correspondence and semantic segmentation. By modeling long-range interactions, dense CRFs provide a more detailed labelling compared to their sparse counterparts. Variational inference in these dense models is performed using a filtering-based mean-field algorithm in order to obtain a fully-factorized distribution minimising the Kullback-Leibler divergence to the true distribution. In contrast to the continuous relaxation-based energy minimisation algorithms used for sparse CRFs, the mean-field algorithm fails to provide strong theoretical guarantees on the quality of its solutions. To address this deficiency, we show that it is possible to use the same filtering approach to speed-up the optimisation of several continuous relaxations. Specifically, we solve a convex quadratic programming (QP) relaxation using the efficient Frank-Wolfe algorithm. This also allows us to solve difference-of-convex relaxations via the iterative concave-convex procedure where each iteration requires solving a convex QP. Finally, we develop a novel divide-and-conquer method to compute the subgradients of a linear programming relaxation that provides the best theoretical bounds for energy minimisation. We demonstrate the advantage of continuous relaxations over the widely used mean-field algorithm on publicly available datasets.