Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration

TL;DR

This paper introduces a new framework for evaluating the reliability of language models' relational knowledge by analyzing their confidence calibration across multiple modalities, revealing overconfidence and semantic grounding issues.

Contribution

It proposes a calibration probing framework that assesses model confidence in relational knowledge, addressing limitations of traditional accuracy-based evaluations.

Findings

Most models are overconfident, especially those trained with masking objectives.

Confidence estimates that consider statement rephrasing are better calibrated.

Large models struggle to encode semantics of linguistic confidence expressions.

Abstract

Knowledge probing quantifies how much relational knowledge a language model (LM) has acquired during pre-training. Existing knowledge probes evaluate model capabilities through metrics like prediction accuracy and precision. Such evaluations fail to account for the model's reliability, reflected in the calibration of its confidence scores. In this paper, we propose a novel calibration probing framework for relational knowledge, covering three modalities of model confidence: (1) intrinsic confidence, (2) structural consistency and (3) semantic grounding. Our extensive analysis of ten causal and six masked language models reveals that most models, especially those pre-trained with the masking objective, are overconfident. The best-calibrated scores come from confidence estimates that account for inconsistencies due to statement rephrasing. Moreover, even the largest pre-trained models fail to encode the semantics of linguistic confidence expressions accurately.