Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge

TL;DR

This paper investigates how large language models prefer to learn from conflicting data, finding they favor formal, consistent texts, which influences their learning speed and knowledge treatment, with preferences being adjustable.

Contribution

It reveals that LLMs have human-like preferences for formal and consistent data, and demonstrates how these preferences can be manipulated through data consistency.

Findings

LLMs prefer formal texts and those with fewer spelling errors.

Preferences are consistent across models and languages, especially in larger models.

Preferences can be altered by changing data consistency levels.

Abstract

Having been trained on massive pretraining data, large language models have shown excellent performance on many knowledge-intensive tasks. However, pretraining data tends to contain misleading and even conflicting information, and it is intriguing to understand how LLMs handle these noisy data during training. In this study, we systematically analyze LLMs' learning preferences for data with conflicting knowledge. We find that pretrained LLMs establish learning preferences similar to humans, i.e., preferences towards formal texts and texts with fewer spelling errors, resulting in faster learning and more favorable treatment of knowledge in data with such features when facing conflicts. This finding is generalizable across models and languages and is more evident in larger models. An in-depth analysis reveals that LLMs tend to trust data with features that signify consistency with the majority of data, and it is possible to instill new preferences and erase old ones by manipulating the degree of consistency with the majority data.