Variational approach to unsupervised learning

TL;DR

This paper introduces a variational approach to understanding unsupervised learning, specifically focusing on deep belief networks, by leveraging symmetry principles and concepts from physics to explain their efficiency.

Contribution

It presents a novel theoretical framework connecting deep belief networks with symmetry and renormalization group concepts, without relying on prior assumptions.

Findings

Deep belief networks can be understood through symmetry principles.

The approach links physics concepts to machine learning efficiency.

Provides a new theoretical foundation for unsupervised learning methods.

Abstract

Deep belief networks are used extensively for unsupervised stochastic learning on large datasets. Compared to other deep learning approaches their layer-by-layer learning makes them highly scalable. Unfortunately, the principles by which they achieve efficient learning are not well understood. Numerous attempts have been made to explain their efficiency and applicability to a wide class of learning problems in terms of principles drawn from cognitive psychology, statistics, information theory, and more recently physics, but quite often these imported principles lack strong scientific foundation. Here we demonstrate how one can arrive at convolutional deep belief networks as potential solution to unsupervised learning problems without making assumptions about the underlying framework. To do this, we exploit the notion of symmetry that is fundamental in machine learning, physics and other fields, utilizing the particular form of the functional renormalization group in physics.