A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems

TL;DR

This paper provides a comprehensive empirical comparison of 32 modern inference techniques for complex energy minimization problems in computer vision, highlighting the effectiveness of polyhedral methods and integer programming in challenging models.

Contribution

It modernizes and expands previous studies by evaluating inference methods on a large, diverse set of real-world energy minimization instances using the OpenGM 2 framework.

Findings

Polyhedral methods and integer programming are competitive in runtime and solution quality.

Results largely agree with past studies for traditional models, but differ for complex, modern models.

Extensive empirical data supports recommendations for inference techniques in diverse applications.

Abstract

Szeliski et al. published an influential study in 2006 on energy minimization methods for Markov Random Fields (MRF). This study provided valuable insights in choosing the best optimization technique for certain classes of problems. While these insights remain generally useful today, the phenomenal success of random field models means that the kinds of inference problems that have to be solved changed significantly. Specifically, the models today often include higher order interactions, flexible connectivity structures, large la\-bel-spaces of different cardinalities, or learned energy tables. To reflect these changes, we provide a modernized and enlarged study. We present an empirical comparison of 32 state-of-the-art optimization techniques on a corpus of 2,453 energy minimization instances from diverse applications in computer vision. To ensure reproducibility, we evaluate all methods in the OpenGM 2 framework and report extensive results regarding runtime and solution quality. Key insights from our study agree with the results of Szeliski et al. for the types of models they studied. However, on new and challenging types of models our findings disagree and suggest that polyhedral methods and integer programming solvers are competitive in terms of runtime and solution quality over a large range of model types.