Coarse-to-Fine Lifted MAP Inference in Computer Vision

TL;DR

This paper introduces a coarse-to-fine lifted inference framework for computer vision that improves efficiency and anytime performance of MAP inference algorithms without sacrificing solution quality.

Contribution

It presents a generic C2F inference template that applies lifted inference to CV problems, enabling more efficient and scalable MAP solutions.

Findings

Lifted algorithms outperform flat algorithms in anytime performance.

The approach maintains solution quality while reducing computation time.

Lifted inference is effectively applied to stereo vision and image segmentation.

Abstract

There is a vast body of theoretical research on lifted inference in probabilistic graphical models (PGMs). However, few demonstrations exist where lifting is applied in conjunction with top of the line applied algorithms. We pursue the applicability of lifted inference for computer vision (CV), with the insight that a globally optimal (MAP) labeling will likely have the same label for two symmetric pixels. The success of our approach lies in efficiently handling a distinct unary potential on every node (pixel), typical of CV applications. This allows us to lift the large class of algorithms that model a CV problem via PGM inference. We propose a generic template for coarse-to-fine (C2F) inference in CV, which progressively refines an initial coarsely lifted PGM for varying quality-time trade-offs. We demonstrate the performance of C2F inference by developing lifted versions of two near state-of-the-art CV algorithms for stereo vision and interactive image segmentation. We find that, against flat algorithms, the lifted versions have a much superior anytime performance, without any loss in final solution quality.