Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications

TL;DR

This paper investigates how compression affects LLMs' knowledge, proposing that knowledge is displaced rather than forgotten, and introduces an inference-time dynamic prompting method that effectively restores performance with efficiency gains.

Contribution

The study introduces the IDP method for knowledge recovery in compressed LLMs, demonstrating its advantages over re-training and providing insights into knowledge displacement.

Findings

Prompting with IDP outperforms re-training methods like LoRA.

Knowledge is displaced, not forgotten, after compression.

IDP reduces inference latency by 60% and parameter size by 21x.

Abstract

Compressing Large Language Models (LLMs) often leads to reduced performance, especially for knowledge-intensive tasks. In this work, we dive into how compression damages LLMs' inherent knowledge and the possible remedies. We start by proposing two conjectures on the nature of the damage: one is certain knowledge being forgotten (or erased) after LLM compression, hence necessitating the compressed model to (re)learn from data with additional parameters; the other presumes that knowledge is internally displaced and hence one requires merely "inference re-direction" with input-side augmentation such as prompting, to recover the knowledge-related performance. Extensive experiments are then designed to (in)validate the two conjectures. We observe the promise of prompting in comparison to model tuning; we further unlock prompting's potential by introducing a variant called Inference-time Dynamic Prompting (IDP), that can effectively increase prompt diversity without incurring any inference overhead. Our experiments consistently suggest that compared to the classical re-training alternatives such as LoRA, prompting with IDP leads to better or comparable post-compression performance recovery, while saving the extra parameter size by 21x and reducing inference latency by 60%. Our experiments hence strongly endorse the conjecture of "knowledge displaced" over "knowledge forgotten", and shed light on a new efficient mechanism to restore compressed LLM performance. We additionally visualize and analyze the different attention and activation patterns between prompted and re-trained models, demonstrating they achieve performance recovery in two different regimes.