Deep learning systems as complex networks

TL;DR

This paper explores deep belief networks through complex network analysis techniques to understand their structural and functional properties, aiming to shed light on their emergent dynamics and internal mechanisms.

Contribution

It introduces a novel approach of applying complex network analysis to deep belief networks to analyze their structure and function.

Findings

Revealed structural properties of deep belief networks

Provided insights into their emergent dynamics

Linked network structure to learning capabilities

Abstract

Thanks to the availability of large scale digital datasets and massive amounts of computational power, deep learning algorithms can learn representations of data by exploiting multiple levels of abstraction. These machine learning methods have greatly improved the state-of-the-art in many challenging cognitive tasks, such as visual object recognition, speech processing, natural language understanding and automatic translation. In particular, one class of deep learning models, known as deep belief networks, can discover intricate statistical structure in large data sets in a completely unsupervised fashion, by learning a generative model of the data using Hebbian-like learning mechanisms. Although these self-organizing systems can be conveniently formalized within the framework of statistical mechanics, their internal functioning remains opaque, because their emergent dynamics cannot be solved analytically. In this article we propose to study deep belief networks using techniques commonly employed in the study of complex networks, in order to gain some insights into the structural and functional properties of the computational graph resulting from the learning process.