Structural Analysis of Sparse Neural Networks

TL;DR

This paper explores the structural properties of sparse neural networks, embedding arbitrary graph structures into ANNs, and analyzing how these structures influence performance, aiming to enhance understanding and design of neural architectures.

Contribution

It introduces a method to embed arbitrary directed acyclic graphs into neural networks and studies their performance prediction based on structural properties.

Findings

Structural properties can predict network performance.

Embedding arbitrary graphs into ANNs is feasible.

Insights may guide neural architecture design.

Abstract

Sparse Neural Networks regained attention due to their potential for mathematical and computational advantages. We give motivation to study Artificial Neural Networks (ANNs) from a network science perspective, provide a technique to embed arbitrary Directed Acyclic Graphs into ANNs and report study results on predicting the performance of image classifiers based on the structural properties of the networks' underlying graph. Results could further progress neuroevolution and add explanations for the success of distinct architectures from a structural perspective.