Why bother with geometry? On the relevance of linear decompositions of Transformer embeddings

TL;DR

This paper investigates whether linear decompositions of Transformer embeddings are meaningful by analyzing their correlation with model performance and variability across different training runs.

Contribution

It provides empirical evidence that geometric decompositions reflect model-specific traits more than sentence-specific information, highlighting variability across training runs.

Findings

Decomposition indicators correlate with model performance.

High variability suggests geometry reflects model-specific traits.

Similar training conditions do not produce similar embedding spaces.

Abstract

A recent body of work has demonstrated that Transformer embeddings can be linearly decomposed into well-defined sums of factors, that can in turn be related to specific network inputs or components. There is however still a dearth of work studying whether these mathematical reformulations are empirically meaningful. In the present work, we study representations from machine-translation decoders using two of such embedding decomposition methods. Our results indicate that, while decomposition-derived indicators effectively correlate with model performance, variation across different runs suggests a more nuanced take on this question. The high variability of our measurements indicate that geometry reflects model-specific characteristics more than it does sentence-specific computations, and that similar training conditions do not guarantee similar vector spaces.