Semi-Supervised Learning -- A Statistical Physics Approach

TL;DR

This paper introduces a statistical physics-based semi-supervised learning method that estimates classification distributions using a Multicanonical Markov chain Monte-Carlo algorithm, improving robustness and flexibility over traditional energy minimization techniques.

Contribution

It proposes a novel probabilistic approach to semi-supervised learning that generalizes existing energy-based methods by estimating classification distributions instead of minimal cuts.

Findings

More accurate classification results on toy and real gene expression data.

Robustness to unseen class types through soft class assignments.

Flexible application to various energy functions in semi-supervised learning.

Abstract

We present a novel approach to semi-supervised learning which is based on statistical physics. Most of the former work in the field of semi-supervised learning classifies the points by minimizing a certain energy function, which corresponds to a minimal k-way cut solution. In contrast to these methods, we estimate the distribution of classifications, instead of the sole minimal k-way cut, which yields more accurate and robust results. Our approach may be applied to all energy functions used for semi-supervised learning. The method is based on sampling using a Multicanonical Markov chain Monte-Carlo algorithm, and has a straightforward probabilistic interpretation, which allows for soft assignments of points to classes, and also to cope with yet unseen class types. The suggested approach is demonstrated on a toy data set and on two real-life data sets of gene expression.