Boltzmann-Machine Learning of Prior Distributions of Binarized Natural Images

TL;DR

This paper uses Boltzmann machines to learn prior distributions of binarized natural images, revealing structured interactions with sublattice patterns and characteristic length scales, validated by mean-field, Bethe, and Monte Carlo methods.

Contribution

It introduces a novel application of Boltzmann machines to model image priors, uncovering universal interaction patterns and validating them with multiple computational approaches.

Findings

Emergence of two-sublattice interaction structure

Universal interaction length scale of approximately four lattice spacings

Validation of results across mean-field, Bethe, and Monte Carlo methods

Abstract

Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflects the individual characteristics of the three sets of pictures that we process. Meanwhile, in a longer spatial scale, a longer-range, although still rapidly decaying, ferromagnetic interaction commonly appears in all cases. The characteristic length scale of the interactions is universally up to approximately four lattice spacings $\xi \approx 4$. These results are derived by using the mean-field method, which effectively reduces the computational time required in a Boltzmann machine. An improved mean-field method called the Bethe approximation also gives the same results, as well as the Monte Carlo method does for small size images. These reinforce the validity of our analysis and findings. Relations to criticality, frustration, and simple-cell receptive fields are also discussed.