Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctness

TL;DR

This paper investigates whether large language models have privileged knowledge about answer correctness, finding domain-specific advantages in factual knowledge but not in math reasoning, and localizing this advantage within model layers.

Contribution

It introduces a method to disentangle genuine privileged knowledge in LLMs by analyzing disagreement subsets and localizes the domain-specific advantage across model layers.

Findings

Self-representations outperform peer representations in factual knowledge tasks.

No advantage of self-representations in math reasoning tasks.

Privileged knowledge in factual tasks emerges from early-to-mid layers.

Abstract

Humans use introspection to evaluate their understanding through private internal states inaccessible to external observers. We investigate whether large language models possess similar privileged knowledge about answer correctness, information unavailable through external observation. We train correctness classifiers on question representations from both a model's own hidden states and external models, testing whether self-representations provide a performance advantage. On standard evaluation, we find no advantage: self-probes perform comparably to peer-model probes. We hypothesize this is due to high inter-model agreement of answer correctness. To isolate genuine privileged knowledge, we evaluate on disagreement subsets, where models produce conflicting predictions. Here, we discover domain-specific privileged knowledge: self-representations consistently outperform peer representations in factual knowledge tasks, but show no advantage in math reasoning. We further localize this domain asymmetry across model layers, finding that the factual advantage emerges progressively from early-to-mid layers onward, consistent with model-specific memory retrieval, while math reasoning shows no consistent advantage at any depth.