Omitted Variable Bias in Language Models Under Distribution Shift

TL;DR

This paper investigates how unobserved variables cause bias in language model evaluation under distribution shifts and proposes a framework to bound worst-case performance considering these hidden factors.

Contribution

It introduces a novel framework linking unobserved variables to performance bounds, improving out-of-distribution evaluation and optimization of language models.

Findings

Bounds on worst-case performance under distribution shift.

Improved out-of-distribution evaluation methods.

Inference about unobserved variables when labels are available.

Abstract

Despite their impressive performance on a wide variety of tasks, modern language models remain susceptible to distribution shifts, exhibiting brittle behavior when evaluated on data that differs in distribution from their training data. In this paper, we describe how distribution shifts in language models can be separated into observable and unobservable components, and we discuss how established approaches for dealing with distribution shift address only the former. Importantly, we identify that the resulting omitted variable bias from unobserved variables can compromise both evaluation and optimization in language models. To address this challenge, we introduce a framework that maps the strength of the omitted variables to bounds on the worst-case generalization performance of language models under distribution shift. In empirical experiments, we show that using these bounds directly in language model evaluation and optimization provides more principled measures of out-of-distribution performance, improves true out-of-distribution performance relative to standard distribution shift adjustment methods, and further enables inference about the strength of the omitted variables when target distribution labels are available.