Large Margin Boltzmann Machines and Large Margin Sigmoid Belief Networks

TL;DR

This paper introduces large margin probabilistic graphical models, LMBMs and LMSBNs, enabling fast, accurate structured prediction for complex graph structures, overcoming previous computational limitations.

Contribution

The paper proposes LMBMs and LMSBNs, novel models that enable efficient, high-performance structured prediction on complex graphs, surpassing existing methods in speed and accuracy.

Findings

LMSBNs allow polynomial-time inference with high probability.

The models outperform state-of-the-art methods in multi-label scene classification.

Significant performance gains demonstrated on complex graph structures.

Abstract

Current statistical models for structured prediction make simplifying assumptions about the underlying output graph structure, such as assuming a low-order Markov chain, because exact inference becomes intractable as the tree-width of the underlying graph increases. Approximate inference algorithms, on the other hand, force one to trade off representational power with computational efficiency. In this paper, we propose two new types of probabilistic graphical models, large margin Boltzmann machines (LMBMs) and large margin sigmoid belief networks (LMSBNs), for structured prediction. LMSBNs in particular allow a very fast inference algorithm for arbitrary graph structures that runs in polynomial time with a high probability. This probability is data-distribution dependent and is maximized in learning. The new approach overcomes the representation-efficiency trade-off in previous models and allows fast structured prediction with complicated graph structures. We present results from applying a fully connected model to multi-label scene classification and demonstrate that the proposed approach can yield significant performance gains over current state-of-the-art methods.