Measuring the metacognition of AI

TL;DR

This paper advocates for using the meta-d' framework and signal detection theory to measure AI metacognition, demonstrating their application on large language models to assess confidence and decision regulation.

Contribution

It introduces the meta-d' framework as a standard for measuring AI metacognitive sensitivity and applies psychophysical methods to evaluate LLMs' decision confidence and risk regulation.

Findings

Meta-d' enables comparison of LLMs' metacognitive sensitivity.

SDT reveals LLMs' increased conservatism under high risk.

Experiments on GPT-5, DeepSeek-V3.2-Exp, and Mistral-Medium-2508 demonstrate practical utility.

Abstract

A robust decision-making process must take into account uncertainty, especially when the choice involves inherent risks. Because artificial Intelligence (AI) systems are increasingly integrated into decision-making workflows, managing uncertainty relies more and more on the metacognitive capabilities of these systems; i.e, their ability to assess the reliability of and regulate their own decisions. Hence, it is crucial to employ robust methods to measure the metacognitive abilities of AI. This paper is primarily a methodological contribution arguing for the adoption of the meta-d' framework as the gold standard for assessing the metacognitive sensitivity of AIs--the ability to generate confidence ratings that distinguish correct from incorrect responses. Moreover, we propose to leverage signal detection theory (SDT) to measure the ability of AIs to spontaneously regulate their decisions based on uncertainty and risk. To demonstrate the practical utility of these psychophysical frameworks, we conduct two series of experiments on three large language models (LLMs)--GPT-5, DeepSeek-V3.2-Exp, and Mistral-Medium-2508. In the first experiments, LLMs performed a primary judgment followed by a confidence rating. In the second, LLMs only performed the primary judgment, while we manipulated the risk associated with either response. On the one hand, applying the meta-d' framework allows us to conduct comparisons along three axes: comparing an LLM to optimality, comparing different LLMs on a given task, and comparing the same LLM across different tasks. On the other hand, SDT allows us to assess whether LLMs become more conservative when risk is high.