Knowledge Boundary Discovery for Large Language Models

TL;DR

This paper introduces Knowledge Boundary Discovery (KBD), a reinforcement learning framework that automatically identifies the knowledge limits of large language models by generating answerable and unanswerable questions, aiding in their evaluation.

Contribution

The paper presents a novel RL-based method to automatically discover LLMs' knowledge boundaries through question generation and environment exploration, addressing hallucination issues.

Findings

KBD effectively finds knowledge boundaries comparable to manual datasets.

KBD-generated questions match human-crafted benchmarks in quality.

The approach provides a new way to evaluate LLMs' knowledge limits.

Abstract

We propose Knowledge Boundary Discovery (KBD), a reinforcement learning based framework to explore the knowledge boundaries of the Large Language Models (LLMs). We define the knowledge boundary by automatically generating two types of questions: (i) those the LLM can confidently answer (within-knowledge boundary) and (ii) those it cannot (beyond-knowledge boundary). Iteratively exploring and exploiting the LLM's responses to find its knowledge boundaries is challenging because of the hallucination phenomenon. To find the knowledge boundaries of an LLM, the agent interacts with the LLM under the modeling of exploring a partially observable environment. The agent generates a progressive question as the action, adopts an entropy reduction as the reward, receives the LLM's response as the observation and updates its belief states. We demonstrate that the KBD detects knowledge boundaries of LLMs by automatically finding a set of non-trivial answerable and unanswerable questions. We validate the KBD by comparing its generated knowledge boundaries with manually crafted LLM benchmark datasets. Experiments show that our KBD-generated question set is comparable to the human-generated datasets. Our approach paves a new way to evaluate LLMs.