Of Non-Linearity and Commutativity in BERT

TL;DR

This paper investigates the non-linearity and layer interactions in BERT, revealing the importance of feed-forward networks, the fuzzy feature extraction across layers, and BERT's bias towards layer commutativity due to skip connections.

Contribution

It introduces a method to measure non-linearity in transformers and provides new insights into the role of feed-forward networks and layer interactions in BERT.

Findings

FFNs are crucial and cannot be replaced without performance loss

Layers extract features in a fuzzy, non-hierarchical manner

BERT exhibits an inductive bias towards layer commutativity due to skip connections

Abstract

In this work we provide new insights into the transformer architecture, and in particular, its best-known variant, BERT. First, we propose a method to measure the degree of non-linearity of different elements of transformers. Next, we focus our investigation on the feed-forward networks (FFN) inside transformers, which contain 2/3 of the model parameters and have so far not received much attention. We find that FFNs are an inefficient yet important architectural element and that they cannot simply be replaced by attention blocks without a degradation in performance. Moreover, we study the interactions between layers in BERT and show that, while the layers exhibit some hierarchical structure, they extract features in a fuzzy manner. Our results suggest that BERT has an inductive bias towards layer commutativity, which we find is mainly due to the skip connections. This provides a justification for the strong performance of recurrent and weight-shared transformer models.