Faster Algorithms for Max-Product Message-Passing

TL;DR

This paper introduces faster message-passing algorithms for graphical models, reducing computational complexity by exploiting model structures with large cliques and latent variables, applicable to various common models.

Contribution

It presents novel algorithms that decrease the complexity exponent in message-passing by leveraging the structure of large cliques and latent variables.

Findings

Reduced complexity exponent for message-passing algorithms.

Applicable to grids, trees, and ring-structured models.

Improved efficiency in both exact and approximate inference.

Abstract

Maximum A Posteriori inference in graphical models is often solved via message-passing algorithms, such as the junction-tree algorithm, or loopy belief-propagation. The exact solution to this problem is well known to be exponential in the size of the model's maximal cliques after it is triangulated, while approximate inference is typically exponential in the size of the model's factors. In this paper, we take advantage of the fact that many models have maximal cliques that are larger than their constituent factors, and also of the fact that many factors consist entirely of latent variables (i.e., they do not depend on an observation). This is a common case in a wide variety of applications, including grids, trees, and ring-structured models. In such cases, we are able to decrease the exponent of complexity for message-passing by 0.5 for both exact and approximate inference.