Automorphism Groups of Graphical Models and Lifted Variational Inference

TL;DR

This paper introduces the automorphism group of graphical models to formalize symmetry, enabling lifted variational inference that reduces computational complexity and improves bounds in MAP inference.

Contribution

It formalizes the automorphism group of exponential families, providing a unified framework for lifted inference and developing the first lifted variational algorithm with tighter bounds.

Findings

Automorphism groups partition variables into equivalent classes (orbits).

Lifted inference reduces complexity by operating on orbits instead of individual variables.

The proposed lifted variational inference algorithm achieves tighter bounds than previous methods.

Abstract

Using the theory of group action, we first introduce the concept of the automorphism group of an exponential family or a graphical model, thus formalizing the general notion of symmetry of a probabilistic model. This automorphism group provides a precise mathematical framework for lifted inference in the general exponential family. Its group action partitions the set of random variables and feature functions into equivalent classes (called orbits) having identical marginals and expectations. Then the inference problem is effectively reduced to that of computing marginals or expectations for each class, thus avoiding the need to deal with each individual variable or feature. We demonstrate the usefulness of this general framework in lifting two classes of variational approximation for maximum a posteriori (MAP) inference: local linear programming (LP) relaxation and local LP relaxation with cycle constraints; the latter yields the first lifted variational inference algorithm that operates on a bound tighter than the local constraints.