Variational Chernoff Bounds for Graphical Models

TL;DR

This paper develops a method to compute rigorous upper and lower bounds on event probabilities in graphical models using generalized Chernoff bounds, complementing existing heuristic estimates with provable guarantees.

Contribution

It introduces a convex optimization-based approach to derive rigorous probability bounds in graphical models, extending beyond marginal estimates.

Findings

The method provides useful, rigorous bounds on event probabilities.

The approach has comparable computational cost to existing variational methods.

Simulations demonstrate the effectiveness of the bounds.

Abstract

Recent research has made significant progress on the problem of bounding log partition functions for exponential family graphical models. Such bounds have associated dual parameters that are often used as heuristic estimates of the marginal probabilities required in inference and learning. However these variational estimates do not give rigorous bounds on marginal probabilities, nor do they give estimates for probabilities of more general events than simple marginals. In this paper we build on this recent work by deriving rigorous upper and lower bounds on event probabilities for graphical models. Our approach is based on the use of generalized Chernoff bounds to express bounds on event probabilities in terms of convex optimization problems; these optimization problems, in turn, require estimates of generalized log partition functions. Simulations indicate that this technique can result in useful, rigorous bounds to complement the heuristic variational estimates, with comparable computational cost.