What Matters in Transformers? Not All Attention is Needed

TL;DR

This paper investigates redundancy in Transformer models, revealing that many attention layers can be pruned without significant performance loss, leading to more efficient architectures.

Contribution

It introduces a similarity-based metric to analyze redundancy across Transformer modules and proposes a joint layer dropping method for improved efficiency.

Findings

High redundancy in attention layers allows pruning without performance loss.

Pruning half of attention layers yields 48.4% speedup with only 2.4% performance drop.

Joint dropping of attention and MLP layers can retain 90% performance while removing 31 layers.

Abstract

While scaling Transformer-based large language models (LLMs) has demonstrated promising performance across various tasks, it also introduces redundant architectures, posing efficiency challenges for real-world deployment. Despite some recognition of redundancy in LLMs, the variability of redundancy across different architectures in transformers, such as MLP and Attention layers, is under-explored. In this work, we investigate redundancy across different modules within Transformers, including Blocks, MLP, and Attention layers, using a similarity-based metric. Surprisingly, despite the critical role of attention layers in distinguishing transformers from other architectures, we found that a large portion of these layers exhibit excessively high similarity and can be pruned without degrading performance. For instance, Llama-2-70B achieved a 48.4\% speedup with only a 2.4\% performance drop by pruning half of the attention layers. Furthermore, by tracing model checkpoints throughout the training process, we observed that attention layer redundancy is inherent and consistent across training stages. Additionally, we further propose a method that jointly drops Attention and MLP layers, allowing us to more aggressively drop additional layers. For instance, when dropping 31 layers (Attention + MLP), Llama-2-13B still retains 90\% of the performance on the MMLU task. Our work provides valuable insights for future network architecture design. The code is released at: \url{https://github.com/Shwai-He/LLM-Drop}.