Converting MLPs into Polynomials in Closed Form

TL;DR

This paper derives closed-form polynomial approximations of MLPs and GLUs, enabling interpretability through linear algebra and revealing how network complexity evolves during training.

Contribution

It provides the first theoretical derivation of optimal polynomial approximations for MLPs and GLUs, facilitating interpretability and analysis of neural network complexity.

Findings

Quadratic approximants explain over 95% of variance in outputs.

Eigen-decomposition of approximants aids interpretability.

Networks become more complex during training, starting simple.

Abstract

Recent work has shown that purely quadratic functions can replace MLPs in transformers with no significant loss in performance, while enabling new methods of interpretability based on linear algebra. In this work, we theoretically derive closed-form least-squares optimal approximations of feedforward networks (multilayer perceptrons and gated linear units) using polynomial functions of arbitrary degree. When the $R^2$ is high, this allows us to interpret MLPs and GLUs by visualizing the eigendecomposition of the coefficients of their linear and quadratic approximants. We also show that these approximants can be used to create SVD-based adversarial examples. By tracing the $R^2$ of linear and quadratic approximants across training time, we find new evidence that networks start out simple, and get progressively more complex. Even at the end of training, however, our quadratic approximants explain over 95% of the variance in network outputs.