Multi-level Data Representation For Training Deep Helmholtz Machines

TL;DR

This paper introduces a multi-level data representation approach to enhance the training of biologically plausible deep generative models, specifically Helmholtz Machines, leading to improved image quality and diversity.

Contribution

It proposes a novel multi-level data representation method to improve learning in deep Helmholtz Machines, inspired by human visual perception mechanisms.

Findings

Improved overall image quality in generated samples.

Increased diversity of generated images.

Enhanced utilization of network depth in learning.

Abstract

A vast majority of the current research in the field of Machine Learning is done using algorithms with strong arguments pointing to their biological implausibility such as Backpropagation, deviating the field's focus from understanding its original organic inspiration to a compulsive search for optimal performance. Yet, there have been a few proposed models that respect most of the biological constraints present in the human brain and are valid candidates for mimicking some of its properties and mechanisms. In this paper, we will focus on guiding the learning of a biologically plausible generative model called the Helmholtz Machine in complex search spaces using a heuristic based on the Human Image Perception mechanism. We hypothesize that this model's learning algorithm is not fit for Deep Networks due to its Hebbian-like local update rule, rendering it incapable of taking full advantage of the compositional properties that multi-layer networks provide. We propose to overcome this problem, by providing the network's hidden layers with visual queues at different resolutions using a Multi-level Data representation. The results on several image datasets showed the model was able to not only obtain better overall quality but also a wider diversity in the generated images, corroborating our intuition that using our proposed heuristic allows the model to take more advantage of the network's depth growth. More importantly, they show the unexplored possibilities underlying brain-inspired models and techniques.