Riemann-Theta Boltzmann Machine

TL;DR

This paper introduces a novel Boltzmann machine with continuous visible and discrete hidden states, utilizing Riemann-Theta functions for analytical solutions, and demonstrates its application in neural networks with enhanced modeling capacity.

Contribution

It presents a new Boltzmann machine model with analytical solutions involving Riemann-Theta functions and applies the conditional expectation as an activation function in neural networks.

Findings

Analytical density function involving Riemann-Theta functions.

Conditional expectation as an activation function.

Successful training with standard optimization techniques.

Abstract

A general Boltzmann machine with continuous visible and discrete integer valued hidden states is introduced. Under mild assumptions about the connection matrices, the probability density function of the visible units can be solved for analytically, yielding a novel parametric density function involving a ratio of Riemann-Theta functions. The conditional expectation of a hidden state for given visible states can also be calculated analytically, yielding a derivative of the logarithmic Riemann-Theta function. The conditional expectation can be used as activation function in a feedforward neural network, thereby increasing the modelling capacity of the network. Both the Boltzmann machine and the derived feedforward neural network can be successfully trained via standard gradient- and non-gradient-based optimization techniques.