Cognitive bias in large language models: Cautious optimism meets anti-Panglossian meliorism

TL;DR

This paper explores cognitive biases in large language models, highlighting both the presence of biases and the potential for reducing them, with philosophical insights on human rationality and data influence.

Contribution

It introduces a nuanced view of biases in language models, emphasizing cautious optimism and the importance of addressing genuine biases beyond fairness concerns.

Findings

Models exhibit genuine cognitive biases.

Bias reduction efforts are feasible and necessary.

Philosophical implications for human rationality.

Abstract

Traditional discussions of bias in large language models focus on a conception of bias closely tied to unfairness, especially as affecting marginalized groups. Recent work raises the novel possibility of assessing the outputs of large language models for a range of cognitive biases familiar from research in judgment and decisionmaking. My aim in this paper is to draw two lessons from recent discussions of cognitive bias in large language models: cautious optimism about the prevalence of bias in current models coupled with an anti-Panglossian willingness to concede the existence of some genuine biases and work to reduce them. I draw out philosophical implications of this discussion for the rationality of human cognitive biases as well as the role of unrepresentative data in driving model biases.