Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Models

TL;DR

This paper introduces a meta-cognitive framework for knowledge augmentation in Large Language Models, aiming to improve their ability to distinguish between known, confused, and missing knowledge, thereby reducing errors and enhancing calibration.

Contribution

It presents a novel meta-cognitive approach that uses internal signals to guide targeted knowledge expansion and aligns certainty with accuracy, improving LLM reliability.

Findings

Outperforms strong baselines in knowledge tasks

Enhances model calibration and confidence accuracy

Effectively distinguishes knowns from unknowns

Abstract

Knowledge augmentation has significantly enhanced the performance of Large Language Models (LLMs) in knowledge-intensive tasks. However, existing methods typically operate on the simplistic premise that model performance equates with internal knowledge, overlooking the knowledge-confidence gaps that lead to overconfident errors or uncertain truths. To bridge this gap, we propose a novel meta-cognitive framework for reliable knowledge augmentation via differentiated intervention and alignment. Our approach leverages internal cognitive signals to partition the knowledge space into mastered, confused, and missing regions, guiding targeted knowledge expansion. Furthermore, we introduce a cognitive consistency mechanism to synchronize subjective certainty with objective accuracy, ensuring calibrated knowledge boundaries. Extensive experiments demonstrate the our framework consistently outperforms strong baselines, validating its rationality in not only enhancing knowledge capabilities but also fostering cognitive behaviors that better distinguish knowns from unknowns.