Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data

TL;DR

This paper introduces a robust evaluation benchmark for honesty in large language models and proposes a novel method to leverage pretraining data to improve their honesty, reducing hallucinations.

Contribution

The paper presents a new benchmark dataset for evaluating honesty in LLMs and a novel approach to utilize pretraining data for enhancing model honesty.

Findings

New benchmark dataset for honesty evaluation.

Pretraining data utilization improves honesty in LLMs.

Reduction in hallucination occurrences.

Abstract

Large language models (LLMs) are highly capable of answering questions, but they are often unaware of their own knowledge boundary, i.e., knowing what they know and what they don't know. As a result, they can generate factually incorrect responses on topics they do not have enough knowledge of, commonly known as hallucination. Rather than hallucinating, a language model should be more honest and respond with "I don't know" when it does not have enough knowledge about a topic. Many methods have been proposed to improve LLM honesty, but their evaluations lack robustness, as they do not take into account the knowledge that the LLM has ingested during its pretraining. In this paper, we propose a more robust evaluation benchmark dataset for LLM honesty by utilizing Pythia, a truly open LLM with publicly available pretraining data. In addition, we also propose a novel method for harnessing the pretraining data to build a more honest LLM.