Transformers learn through gradual rank increase

TL;DR

This paper investigates how transformers learn gradually by increasing the rank of weight differences, providing theoretical proof under simplified conditions and empirical evidence in real-world scenarios.

Contribution

It introduces a rigorous analysis of incremental learning dynamics in transformers, highlighting rank increase as a key aspect of their training process.

Findings

Rank difference between trained and initial weights increases during training

Theoretical proof under diagonal weights and small initialization

Empirical evidence shows the phenomenon occurs in practice

Abstract

We identify incremental learning dynamics in transformers, where the difference between trained and initial weights progressively increases in rank. We rigorously prove this occurs under the simplifying assumptions of diagonal weight matrices and small initialization. Our experiments support the theory and also show that phenomenon can occur in practice without the simplifying assumptions.