Simplicity Bias in 1-Hidden Layer Neural Networks

TL;DR

This paper rigorously defines and demonstrates the presence of simplicity bias in one-hidden-layer neural networks, showing they rely on low-dimensional input projections and proposing an ensemble method to improve robustness.

Contribution

It provides a formal definition of simplicity bias, proves its occurrence in linearly separable data, empirically verifies it on real datasets, and introduces an ensemble approach to enhance robustness.

Findings

Neural networks depend on low-dimensional input projections.

Simplicity bias occurs even with complex features present.

Ensemble training on diverse features improves robustness.

Abstract

Recent works have demonstrated that neural networks exhibit extreme simplicity bias(SB). That is, they learn only the simplest features to solve a task at hand, even in the presence of other, more robust but more complex features. Due to the lack of a general and rigorous definition of features, these works showcase SB on semi-synthetic datasets such as Color-MNIST, MNIST-CIFAR where defining features is relatively easier. In this work, we rigorously define as well as thoroughly establish SB for one hidden layer neural networks. More concretely, (i) we define SB as the network essentially being a function of a low dimensional projection of the inputs (ii) theoretically, we show that when the data is linearly separable, the network primarily depends on only the linearly separable ($1$-dimensional) subspace even in the presence of an arbitrarily large number of other, more complex features which could have led to a significantly more robust classifier, (iii) empirically, we show that models trained on real datasets such as Imagenette and Waterbirds-Landbirds indeed depend on a low dimensional projection of the inputs, thereby demonstrating SB on these datasets, iv) finally, we present a natural ensemble approach that encourages diversity in models by training successive models on features not used by earlier models, and demonstrate that it yields models that are significantly more robust to Gaussian noise.