Efficient SDP Inference for Fully-connected CRFs Based on Low-rank Decomposition

TL;DR

This paper introduces a scalable and efficient algorithm for inference in fully-connected CRFs using low-rank matrix approximation and a tailored quasi-Newton method, enabling applications like pixel-level image co-segmentation.

Contribution

The paper presents a novel, general inference algorithm for fully-connected CRFs based on low-rank decomposition and a specialized SDP solver, improving over previous methods.

Findings

Able to perform inference on previously unsolvable fully-connected CRFs

Demonstrates effectiveness on pixel-level image co-segmentation

Offers a scalable solution with competitive accuracy

Abstract

Conditional Random Fields (CRF) have been widely used in a variety of computer vision tasks. Conventional CRFs typically define edges on neighboring image pixels, resulting in a sparse graph such that efficient inference can be performed. However, these CRFs fail to model long-range contextual relationships. Fully-connected CRFs have thus been proposed. While there are efficient approximate inference methods for such CRFs, usually they are sensitive to initialization and make strong assumptions. In this work, we develop an efficient, yet general algorithm for inference on fully-connected CRFs. The algorithm is based on a scalable SDP algorithm and the low- rank approximation of the similarity/kernel matrix. The core of the proposed algorithm is a tailored quasi-Newton method that takes advantage of the low-rank matrix approximation when solving the specialized SDP dual problem. Experiments demonstrate that our method can be applied on fully-connected CRFs that cannot be solved previously, such as pixel-level image co-segmentation.