Singular Value Representation: A New Graph Perspective On Neural Networks

TL;DR

This paper introduces the Singular Value Representation (SVR), a novel graph-based method for analyzing neural networks' internal states through SVD, revealing new insights into layer interactions and normalization effects.

Contribution

The paper presents SVR, a new spectral graph approach to interpret neural networks, including a statistical framework for identifying meaningful spectral neuron connections.

Findings

Discovery of a dominant multi-layer connection in VGG networks.

Observation of batch normalization inducing significant spectral neuron connections.

Identification of spontaneous sparsification phenomena in deep layers.

Abstract

We introduce the Singular Value Representation (SVR), a new method to represent the internal state of neural networks using SVD factorization of the weights. This construction yields a new weighted graph connecting what we call spectral neurons, that correspond to specific activation patterns of classical neurons. We derive a precise statistical framework to discriminate meaningful connections between spectral neurons for fully connected and convolutional layers. To demonstrate the usefulness of our approach for machine learning research, we highlight two discoveries we made using the SVR. First, we highlight the emergence of a dominant connection in VGG networks that spans multiple deep layers. Second, we witness, without relying on any input data, that batch normalization can induce significant connections between near-kernels of deep layers, leading to a remarkable spontaneous sparsification phenomenon.