In-Context Learning (and Unlearning) of Length Biases

TL;DR

This paper investigates how large language models learn and unlearn length biases in their predictions through in-context learning, revealing their ability to adapt and counteract biases without parameter fine-tuning.

Contribution

It is the first to analyze length biases in in-context learning and demonstrates how models can learn, modulate, and unlearn these biases effectively.

Findings

Models learn length biases in the context window.

Factors influencing bias levels are empirically identified.

In-context learning can counteract length biases without fine-tuning.

Abstract

Large language models have demonstrated strong capabilities to learn in-context, where exemplar input-output pairings are appended to the prompt for demonstration. However, existing work has demonstrated the ability of models to learn lexical and label biases in-context, which negatively impacts both performance and robustness of models. The impact of other statistical data biases remains under-explored, which this work aims to address. We specifically investigate the impact of length biases on in-context learning. We demonstrate that models do learn length biases in the context window for their predictions, and further empirically analyze the factors that modulate the level of bias exhibited by the model. In addition, we show that learning length information in-context can be used to counter the length bias that has been encoded in models (e.g., via fine-tuning). This reveals the power of in-context learning in debiasing model prediction behaviors without the need for costly parameter updates.