On the attribution of confidence to large language models

TL;DR

This paper explores the conceptual and philosophical foundations of attributing confidence levels to large language models, questioning the interpretation, existence, and evaluation of LLM credences.

Contribution

It defends a literal interpretation of LLM credence attributions, discusses their metaphysical plausibility, and highlights epistemic skepticism regarding current evaluation methods.

Findings

LLM credence attributions are likely correctly interpreted as beliefs.

The existence of LLM credences is plausible but not conclusively established.

Current evaluation techniques may not truth-track LLM credences.

Abstract

Credences are mental states corresponding to degrees of confidence in propositions. Attribution of credences to Large Language Models (LLMs) is commonplace in the empirical literature on LLM evaluation. Yet the theoretical basis for LLM credence attribution is unclear. We defend three claims. First, our semantic claim is that LLM credence attributions are (at least in general) correctly interpreted literally, as expressing truth-apt beliefs on the part of scientists that purport to describe facts about LLM credences. Second, our metaphysical claim is that the existence of LLM credences is at least plausible, although current evidence is inconclusive. Third, our epistemic claim is that LLM credence attributions made in the empirical literature on LLM evaluation are subject to non-trivial sceptical concerns. It is a distinct possibility that even if LLMs have credences, LLM credence attributions are generally false because the experimental techniques used to assess LLM credences are not truth-tracking.