Assessing LLM Reasoning Steps via Principal Knowledge Grounding

TL;DR

This paper introduces a comprehensive evaluation suite for assessing how well large language models ground their reasoning in essential knowledge, using a new knowledge repository, specialized metrics, and an efficient evaluator model.

Contribution

It presents a novel framework combining a large-scale knowledge repository, knowledge-grounded metrics, and an evaluator model to systematically assess and improve LLM reasoning groundedness.

Findings

Effective identification of missing or misapplied knowledge in LLMs

Demonstrated integration of metrics into preference optimization

Showcased the framework's ability to reveal reasoning deficiencies

Abstract

Step-by-step reasoning has become a standard approach for large language models (LLMs) to tackle complex tasks. While this paradigm has proven effective, it raises a fundamental question: How can we verify that an LLM's reasoning is accurately grounded in knowledge? To address this question, we introduce a novel evaluation suite that systematically assesses the knowledge grounding of intermediate reasoning. Our framework comprises three key components. (1) Principal Knowledge Collection, a large-scale repository of atomic knowledge essential for reasoning. Based on the collection, we propose (2) knowledge-grounded evaluation metrics designed to measure how well models recall and apply prerequisite knowledge in reasoning. These metrics are computed by our (3) evaluator LLM, a lightweight model optimized for cost-effective and reliable metric computation. Our evaluation suite demonstrates remarkable effectiveness in identifying missing or misapplied knowledge elements, providing crucial insights for uncovering fundamental reasoning deficiencies in LLMs. Beyond evaluation, we demonstrate how these metrics can be integrated into preference optimization, showcasing further applications of knowledge-grounded evaluation.