On Neural Networks as Infinite Tree-Structured Probabilistic Graphical Models

TL;DR

This paper establishes a precise correspondence between deep neural networks and infinite tree-structured probabilistic graphical models, offering new insights into their probabilistic interpretation and inference mechanisms.

Contribution

It introduces a novel framework linking neural networks to infinite tree-structured PGMs, enhancing understanding and enabling new algorithms combining both models' strengths.

Findings

DNNs perform approximate inference akin to PGMs during forward pass

Infinite tree-structured PGMs exactly match neural network computations

Framework facilitates improved interpretation and potential hybrid algorithms

Abstract

Deep neural networks (DNNs) lack the precise semantics and definitive probabilistic interpretation of probabilistic graphical models (PGMs). In this paper, we propose an innovative solution by constructing infinite tree-structured PGMs that correspond exactly to neural networks. Our research reveals that DNNs, during forward propagation, indeed perform approximations of PGM inference that are precise in this alternative PGM structure. Not only does our research complement existing studies that describe neural networks as kernel machines or infinite-sized Gaussian processes, it also elucidates a more direct approximation that DNNs make to exact inference in PGMs. Potential benefits include improved pedagogy and interpretation of DNNs, and algorithms that can merge the strengths of PGMs and DNNs.