Variational Pseudolikelihood for Regularized Ising Inference

TL;DR

This paper introduces a variational method for efficient and accurate maximum pseudolikelihood inference in the Ising model, improving out-of-sample correlation predictions over existing methods.

Contribution

It presents a novel variational algorithm that regularizes the inference by shrinking couplings, enhancing computational efficiency and predictive accuracy.

Findings

Better out-of-sample correlation prediction than existing methods

More computationally efficient inference algorithm

Successfully modeled letter samples from different fonts

Abstract

I propose a variational approach to maximum pseudolikelihood inference of the Ising model. The variational algorithm is more computationally efficient, and does a better job predicting out-of-sample correlations than $L_2$ regularized maximum pseudolikelihood inference as well as mean field and isolated spin pair approximations with pseudocount regularization. The key to the approach is a variational energy that regularizes the inference problem by shrinking the couplings towards zero, while still allowing some large couplings to explain strong correlations. The utility of the variational pseudolikelihood approach is illustrated by training an Ising model to represent the letters A-J using samples of letters from different computer fonts.