Transductive Boltzmann Machines

TL;DR

Transductive Boltzmann Machines (TBMs) introduce a novel transductive learning approach for Gibbs distributions, adaptively constructing minimal sample spaces to improve efficiency and effectiveness over traditional Boltzmann machines.

Contribution

TBMs are the first to achieve transductive learning of Gibbs distributions, overcoming combinatorial challenges by adaptively building minimal sample spaces from data.

Findings

TBMs outperform fully visible Boltzmann machines in efficiency.

TBMs demonstrate superior effectiveness compared to restricted Boltzmann machines.

Theoretical bias-variance analysis supports TBMs' learnability.

Abstract

We present transductive Boltzmann machines (TBMs), which firstly achieve transductive learning of the Gibbs distribution. While exact learning of the Gibbs distribution is impossible by the family of existing Boltzmann machines due to combinatorial explosion of the sample space, TBMs overcome the problem by adaptively constructing the minimum required sample space from data to avoid unnecessary generalization. We theoretically provide bias-variance decomposition of the KL divergence in TBMs to analyze its learnability, and empirically demonstrate that TBMs are superior to the fully visible Boltzmann machines and popularly used restricted Boltzmann machines in terms of efficiency and effectiveness.