On the Information Theoretic Distance Measures and Bidirectional Helmholtz Machines

TL;DR

This paper introduces a generalized Helmholtz machine linked to information theory, demonstrating improved performance over previous models with shallow architectures through theoretical and experimental validation.

Contribution

It establishes a connection between bi-directional Helmholtz machines and information theory, proposing a generalized model that outperforms prior shallow architectures.

Findings

Generalized model outperforms previous ones on shallow architectures

Theoretical analysis supports the effectiveness of the generalized approach

Experimental results confirm improved performance

Abstract

By establishing a connection between bi-directional Helmholtz machines and information theory, we propose a generalized Helmholtz machine. Theoretical and experimental results show that given \textit{shallow} architectures, the generalized model outperforms the previous ones substantially.