Subspace Clustering Based Analysis of Neural Networks

TL;DR

This paper introduces a subspace clustering approach to analyze neural network latent spaces, revealing how input affinities and layer similarities evolve during training and across network architecture.

Contribution

It applies sparse subspace clustering and community detection to neural network analysis, providing new insights into layer-wise convergence and architectural organization.

Findings

Deeper layers show increased input affinity within classes.

Shallower layers converge faster during training.

Network layers reorganize from local to more global similarity as size increases.

Abstract

Tools to analyze the latent space of deep neural networks provide a step towards better understanding them. In this work, we motivate sparse subspace clustering (SSC) with an aim to learn affinity graphs from the latent structure of a given neural network layer trained over a set of inputs. We then use tools from Community Detection to quantify structures present in the input. These experiments reveal that as we go deeper in a network, inputs tend to have an increasing affinity to other inputs of the same class. Subsequently, we utilise matrix similarity measures to perform layer-wise comparisons between affinity graphs. In doing so we first demonstrate that when comparing a given layer currently under training to its final state, the shallower the layer of the network, the quicker it is to converge than the deeper layers. When performing a pairwise analysis of the entire network architecture, we observe that, as the network increases in size, it reorganises from a state where each layer is moderately similar to its neighbours, to a state where layers within a block have high similarity than to layers in other blocks. Finally, we analyze the learned affinity graphs of the final convolutional layer of the network and demonstrate how an input's local neighbourhood affects its classification by the network.