Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slow

TL;DR

This paper introduces APriCoT, a prompting method that reduces biases in language models by encouraging slow, deliberate reasoning, leading to fairer and more accurate predictions.

Contribution

The paper proposes APriCoT, a novel prompting technique that mitigates bias in language models by promoting slow, thoughtful reasoning beyond traditional chain of thought methods.

Findings

APriCoT reduces bias influence in model predictions.

APriCoT improves overall accuracy on MMLU tasks.

Counterfactual prompting alone is insufficient to mitigate bias.

Abstract

Language models are known to absorb biases from their training data, leading to predictions driven by statistical regularities rather than semantic relevance. We investigate the impact of these biases on answer choice preferences in the Massive Multi-Task Language Understanding (MMLU) task. Our findings show that these biases are predictive of model preference and mirror human test-taking strategies even when chain of thought (CoT) reasoning is used. To address this issue, we introduce Counterfactual Prompting with Agnostically Primed CoT (APriCoT). We demonstrate that while Counterfactual Prompting with CoT alone is insufficient to mitigate bias, APriCoT effectively reduces the influence of base-rate probabilities while improving overall accuracy. Our results suggest that mitigating bias requires a slow thinking process which CoT alone may not provide as it tends to reinforce fast thinking model bias under some prompting methodologies. APriCoT is a step toward developing more robust and fair language models that can think slow.