Analyzing the Effect of Noise in LLM Fine-tuning

TL;DR

This study investigates how different types of noise in fine-tuning data affect large language models' behavior, internal representations, and performance across multiple models and NLP tasks.

Contribution

It systematically analyzes the impact of label, grammatical, and typographical noise on LLMs' internal dynamics and task performance, revealing noise-specific effects and localization within the models.

Findings

Label noise causes the most significant performance drop.

Grammatical and typographical noise can sometimes act as regularizers.

Noise effects are mainly localized to task-specific layers, with stable attention patterns.

Abstract

Fine-tuning is the dominant paradigm for adapting pretrained large language models (LLMs) to downstream NLP tasks. In practice, fine-tuning datasets may contain various forms of noise arising from annotation errors, preprocessing artifacts, or automated data collection. While prior work has focused on designing robust learning algorithms to mitigate performance degradation under noisy conditions, comparatively little is known about how different types of noise affect the internal learning dynamics of LLMs during fine-tuning. In this work, we systematically study the impact of noise on model behavior across three pretrained model families (GPT-2, Qwen2 and Llama-2) and three diverse NLP tasks. We introduce controlled perturbations corresponding to three common real-world noise types: label noise, grammatical noise, and typographical noise. Beyond task-level performance, we analyze layer-wise representation changes and attention patterns to understand how noise propagates through the network. Our results show that corrupting labels (i.e. label noise) consistently causes the largest performance degradation, whereas grammatical noise and typographical noise can occasionally yield mild regularization benefits. We further find that noise effects are localized primarily to task-specific layers, while attention structures remain comparatively stable.