Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity

TL;DR

This paper critically evaluates the use of cosine similarity for assessing layer relevance in large language models, demonstrating its limitations and proposing actual performance drop as a more reliable metric.

Contribution

It reveals the inadequacy of cosine similarity as a proxy for layer importance and introduces a more accurate, albeit costly, metric based on performance degradation.

Findings

Cosine similarity often poorly correlates with actual performance loss.

A layer can have low cosine similarity but still be crucial for model performance.

Using actual accuracy drop provides a better assessment of layer importance.

Abstract

Large language models (LLMs) have revolutionized natural language processing. Understanding their internal mechanisms is crucial for developing more interpretable and optimized architectures. Mechanistic interpretability has led to the development of various methods for assessing layer relevance, with cosine similarity being a widely used tool in the field. On this work, we demonstrate that cosine similarity is a poor proxy for the actual performance degradation caused by layer removal. Our theoretical analysis shows that a layer can exhibit an arbitrarily low cosine similarity score while still being crucial to the model's performance. On the other hand, empirical evidence from a range of LLMs confirms that the correlation between cosine similarity and actual performance degradation is often weak or moderate, leading to misleading interpretations of a transformer's internal mechanisms. We propose a more robust metric for assessing layer relevance: the actual drop in model accuracy resulting from the removal of a layer. Even though it is a computationally costly metric, this approach offers a more accurate picture of layer importance, allowing for more informed pruning strategies and lightweight models. Our findings have significant implications for the development of interpretable LLMs and highlight the need to move beyond cosine similarity in assessing layer relevance.