Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative Generation

TL;DR

This paper introduces CAP-TTA, a test-time adaptation method that reduces bias and toxicity in large language models during inference, especially on out-of-distribution prompts, with low latency and improved fluency.

Contribution

It proposes a novel, efficient framework for real-time bias mitigation in LLMs using context-aware updates triggered by bias risk scores.

Findings

CAP-TTA effectively reduces toxicity and bias scores in LLMs.

It achieves lower latency compared to standard optimization methods.

It improves narrative fluency without sacrificing debiasing performance.

Abstract

Although debiased large language models (LLMs) excel at handling known or low-bias prompts, they often fail on unfamiliar and high-bias prompts. We demonstrate via out-of-distribution (OOD) detection that these high-bias prompts cause a distribution shift, degrading static model performance. To enable real-time correction, we propose CAP-TTA, a test-time adaptation framework. CAP-TTA triggers context-aware LoRA updates only when a bias-risk score exceeds a set threshold. By utilizing an offline precomputed diagonal preconditioner, it ensures fast and stable optimization. Across multiple benchmarks and human evaluations, CAP-TTA effectively reduces toxicity/bias score with significantly lower latency than standard optimization methods (e.g., AdamW or SGD). Furthermore, it prevents catastrophic forgetting, and substantially improves narrative fluency over state-of-the-art baselines without compromising debiasing performance.