Bayes-optimal performance in a discrete space

TL;DR

This paper analyzes the theoretical limits of a binary-component unsupervised learning model, deriving the exact Bayes-optimal performance and highlighting the challenges in achieving it through variational methods.

Contribution

It provides an exact calculation of the Bayes-optimal estimator's performance in a discrete space and discusses the limitations of variational approaches.

Findings

Exact Bayes-optimal performance derived

Variational methods generally cannot achieve optimality

Special cases where variational approaches succeed

Abstract

We study a simple model of unsupervised learning where the single symmetry breaking vector has binary components $\pm 1$. We calculate exactly the Bayes-optimal performance of an estimator which is required to lie in the same discrete space. We also show that, except for very special cases, such an estimator cannot be obtained by minimization of a class of variationally optimal potentials.