Deep Neural Networks as Complex Networks

TL;DR

This paper applies Complex Network Theory to model Deep Neural Networks as directed weighted graphs, providing new metrics to analyze their structure and dynamics, which helps distinguish different network architectures and performance levels.

Contribution

It introduces a novel framework using CNT to analyze DNNs as dynamical systems, offering metrics that capture structural and functional differences beyond traditional input-output analysis.

Findings

Metrics discriminate low vs. high performing networks

Framework captures differences in network architecture and parameters

Provides a physically grounded approach to understanding DNN behavior

Abstract

Deep Neural Networks are, from a physical perspective, graphs whose `links` and `vertices` iteratively process data and solve tasks sub-optimally. We use Complex Network Theory (CNT) to represents Deep Neural Networks (DNNs) as directed weighted graphs: within this framework, we introduce metrics to study DNNs as dynamical systems, with a granularity that spans from weights to layers, including neurons. CNT discriminates networks that differ in the number of parameters and neurons, the type of hidden layers and activations, and the objective task. We further show that our metrics discriminate low vs. high performing networks. CNT is a comprehensive method to reason about DNNs and a complementary approach to explain a model's behavior that is physically grounded to networks theory and goes beyond the well-studied input-output relation.