Addressing the Ecological Fallacy in Larger LMs with Human Context

TL;DR

This paper demonstrates that modeling an author's language context during fine-tuning and pre-training enhances the performance of large language models across multiple tasks by addressing the ecological fallacy.

Contribution

It introduces the HuLM and HuFT methods for incorporating author context into large language models, showing improvements over standard fine-tuning and pre-training.

Findings

Fine-tuning with author context improves model performance.

Pre-training with HuLM enhances generalization across tasks.

Modeling author context is crucial for better language understanding.

Abstract

Language model training and inference ignore a fundamental linguistic fact -- there is a dependence between multiple sequences of text written by the same person. Prior work has shown that addressing this form of \textit{ecological fallacy} can greatly improve the performance of multiple smaller (~124M) GPT-based models. In this work, we ask if addressing the ecological fallacy by modeling the author's language context with a specific LM task (called HuLM) can provide similar benefits for a larger-scale model, an 8B Llama model. To this end, we explore variants that process an author's language in the context of their other temporally ordered texts. We study the effect of pre-training with this author context using the HuLM objective, as well as using it during fine-tuning with author context (\textit{HuFT:Human-aware Fine-Tuning}). Empirical comparisons show that addressing the ecological fallacy during fine-tuning alone using QLoRA improves the performance of the larger 8B model over standard fine-tuning. Additionally, QLoRA-based continued HuLM pre-training results in a human-aware model generalizable for improved performance over eight downstream tasks with linear task classifier training alone. These results indicate the utility and importance of modeling language in the context of its original generators, the authors.