Test-Time Fairness and Robustness in Large Language Models
Leonardo Cotta, Chris J. Maddison
TL;DR
This paper introduces a causal, stratified invariance approach to improve test-time fairness and robustness in large language models, effectively reducing biases without additional training.
Contribution
It develops a new stratified invariance concept, a complete observational test, and a data augmentation and prompting strategy for test-time debiasing of LLMs.
Findings
Reduces bias across synthetic and real-world benchmarks
Does not require additional data, finetuning, or pre-training
Guarantees stratified invariance under certain assumptions
Abstract
Frontier Large Language Models (LLMs) can be socially discriminatory or sensitive to spurious features of their inputs. Because only well-resourced corporations can train frontier LLMs, we need robust test-time strategies to control such biases. Existing solutions, which instruct the LLM to be fair or robust, rely on the model's implicit understanding of bias. Causality provides a rich formalism through which we can be explicit about our debiasing requirements. Yet, as we show, a naive application of the standard causal debiasing strategy, counterfactual data augmentation, fails under standard assumptions to debias predictions at an individual level at test time. To address this, we develop a stratified notion of debiasing called stratified invariance, which can capture a range of debiasing requirements from population level to individual level through an additional measurement that…
Peer Reviews
Decision·Submitted to ICLR 2025
The theoretical development of Stratified Invariance and Stratified Data Augmentation is interesting. Also, by experimenting on both synthetic and real-world datasets, the authors demonstrate the advantage of the proposed prompting strategy to boost stratified invariance in LLM predictions at test time.
While the paper’s introduction of "stratified invariance" is an interesting measure of fairness, it appears conceptually close to existing techniques in fair representation learning and causal fairness (e.g., statistical parity). It would be good if the authors could provide an in-depth discussion with other fairness metrics or write out the equations for comparison if this measurement is claimed as a novelty. It is also worth noting that the proposed metric and/or prompting strategy only works
The strength of the paper comes from the clear presentation of the potential issue of directly applying certain causal fairness notions (especially ones that are related to counterfactual invariance) in the LLM context (Section 2), and the attempt to address this issue by proposing stratified invariance (Definition 1), which is a reasonable middle ground between the almost-sure-equality between potential outcomes (counterfactual invariance) and the distribution-level equality (referred to as int
The paper can be improved by (1) considering recent LLM debiasing strategies that do not specifically "rely on model's implicit understanding of bias" (lines 47 -- 49), so that the addressing of the existing LLM literature can be more comprehensive; (2) including discussion on the inference overhead of the proposed pipeline. (1) recent LLM debiasing strategies that do not specifically rely on model's implicit understanding of bias The paper presents criticisms of the existing LLM debiasing str
- Originality: - Unlike previous works that used safety instructions to implicitly address the bias issue, this work leverages the causal invariance framework that utilizes interventions to obtain a less biased result. - This work also developed a stratified invariance notion that is built on observational data (random generations). - A novel OOC strategy is introduced to debias LLM predictions. - Quality: - The theoretical definition and analysis are introduced for stratified
- The presentation of this paper could be substantially improved. I tried very hard to understand this paper, but many things still remain unclear. I will list a few here: - Line 127-138 are helpful for understanding but they only appeared in the method section. I suggest the author can elaborate the problem and objective further in the introduction. - Motivation of applying causal invariance in LLM debiasing is unclear. - It would be better if in Sec. 3 or before, a complete exampl
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques
MethodsCausal inference