A Chain Graph Interpretation of Real-World Neural Networks

TL;DR

This paper introduces a novel interpretation of neural networks as chain graphs, enabling a deeper theoretical understanding and analysis of their structure and inference processes, applicable to complex real-world architectures.

Contribution

It proposes a chain graph framework for neural networks, linking them to probabilistic graphical models and supporting in-depth theoretical analysis of diverse architectures.

Findings

Provides a new theoretical foundation for NNs as chain graphs.

Demonstrates how this interpretation supports analysis of complex NN structures.

Introduces the concept of partially collapsed feed-forward inference.

Abstract

The last decade has witnessed a boom of deep learning research and applications achieving state-of-the-art results in various domains. However, most advances have been established empirically, and their theoretical analysis remains lacking. One major issue is that our current interpretation of neural networks (NNs) as function approximators is too generic to support in-depth analysis. In this paper, we remedy this by proposing an alternative interpretation that identifies NNs as chain graphs (CGs) and feed-forward as an approximate inference procedure. The CG interpretation specifies the nature of each NN component within the rich theoretical framework of probabilistic graphical models, while at the same time remains general enough to cover real-world NNs with arbitrary depth, multi-branching and varied activations, as well as common structures including convolution / recurrent layers, residual block and dropout. We demonstrate with concrete examples that the CG interpretation can provide novel theoretical support and insights for various NN techniques, as well as derive new deep learning approaches such as the concept of partially collapsed feed-forward inference. It is thus a promising framework that deepens our understanding of neural networks and provides a coherent theoretical formulation for future deep learning research.