Towards the Effect of Examples on In-Context Learning: A Theoretical Case Study

TL;DR

This paper provides a theoretical analysis of in-context learning in large language models, quantifying how pre-training knowledge, example label distribution, and noise influence classification accuracy.

Contribution

It introduces a probabilistic model extending Gaussian mixtures to analyze the impact of various factors on ICL performance in binary classification.

Findings

Contradictions between pre-training knowledge and examples affect reliance on each.

Label frequency and noise levels influence prediction accuracy.

Simulations and real-data experiments support theoretical insights.

Abstract

In-context learning (ICL) has emerged as a powerful capability for large language models (LLMs) to adapt to downstream tasks by leveraging a few (demonstration) examples. Despite its effectiveness, the mechanism behind ICL remains underexplored. To better understand how ICL integrates the examples with the knowledge learned by the LLM during pre-training (i.e., pre-training knowledge) and how the examples impact ICL, this paper conducts a theoretical study in binary classification tasks. In particular, we introduce a probabilistic model extending from the Gaussian mixture model to exactly quantify the impact of pre-training knowledge, label frequency, and label noise on the prediction accuracy. Based on our analysis, when the pre-training knowledge contradicts the knowledge in the examples, whether ICL prediction relies more on the pre-training knowledge or the examples depends on the number of examples. In addition, the label frequency and label noise of the examples both affect the accuracy of the ICL prediction, where the minor class has a lower accuracy, and how the label noise impacts the accuracy is determined by the specific noise level of the two classes. Extensive simulations are conducted to verify the correctness of the theoretical results, and real-data experiments also align with the theoretical insights. Our work reveals the role of pre-training knowledge and examples in ICL, offering a deeper understanding of LLMs' behaviors in classification tasks.