On Support Samples of Next Word Prediction

TL;DR

This paper explores the role of support samples in next-word prediction, revealing their intrinsic properties and significance in model generalization, with implications for interpretability and training strategies.

Contribution

It introduces a data-centric interpretability framework using the representer theorem to identify and analyze support samples in language models.

Findings

Support samples are intrinsically identifiable before training.

Non-support samples influence generalization and prevent overfitting.

Their importance increases in deeper model layers.

Abstract

Language models excel in various tasks by making complex decisions, yet understanding the rationale behind these decisions remains a challenge. This paper investigates \emph{data-centric interpretability} in language models, focusing on the next-word prediction task. Using representer theorem, we identify two types of \emph{support samples}-those that either promote or deter specific predictions. Our findings reveal that being a support sample is an intrinsic property, predictable even before training begins. Additionally, while non-support samples are less influential in direct predictions, they play a critical role in preventing overfitting and shaping generalization and representation learning. Notably, the importance of non-support samples increases in deeper layers, suggesting their significant role in intermediate representation formation. These insights shed light on the interplay between data and model decisions, offering a new dimension to understanding language model behavior and interpretability.