Where is the Information in a Deep Neural Network?

TL;DR

This paper explores how information is encoded in deep neural network weights and activations, linking information measures to generalization and invariance, and revealing the impact of architecture and training on learned representations.

Contribution

It introduces a novel measure of information in neural networks based on accuracy and weight complexity, connecting it to generalization and invariance, and relating weight information to activation information.

Findings

Low complexity models generalize better

Models with low complexity learn invariant representations

Information in weights relates to effective information in activations

Abstract

Whatever information a deep neural network has gleaned from training data is encoded in its weights. How this information affects the response of the network to future data remains largely an open question. Indeed, even defining and measuring information entails some subtleties, since a trained network is a deterministic map, so standard information measures can be degenerate. We measure information in a neural network via the optimal trade-off between accuracy of the response and complexity of the weights, measured by their coding length. Depending on the choice of code, the definition can reduce to standard measures such as Shannon Mutual Information and Fisher Information. However, the more general definition allows us to relate information to generalization and invariance, through a novel notion of effective information in the activations of a deep network. We establish a novel relation between the information in the weights and the effective information in the activations, and use this result to show that models with low (information) complexity not only generalize better, but are bound to learn invariant representations of future inputs. These relations hinge not only on the architecture of the model, but also on how it is trained, highlighting the complex inter-dependency between the class of functions implemented by deep neural networks, the loss function used for training them from finite data, and the inductive bias implicit in the optimization.