Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs

TL;DR

The paper introduces SENATOR, a framework that uses structural entropy and Monte Carlo Tree Search to identify and generate targeted synthetic data, improving LLMs' knowledge in specialized domains.

Contribution

It presents a novel Structural Entropy-guided approach combined with MCTS to detect and repair knowledge gaps in LLMs through targeted data generation.

Findings

Enhanced performance on domain-specific benchmarks

Effective detection and repair of knowledge deficiencies

Improved factual accuracy in LLM outputs

Abstract

Large language models (LLMs) have achieved unprecedented performance by leveraging vast pretraining corpora, yet their performance remains suboptimal in knowledge-intensive domains such as medicine and scientific research, where high factual precision is required. While synthetic data provides a promising avenue for augmenting domain knowledge, existing methods frequently generate redundant samples that do not align with the model's true knowledge gaps. To overcome this limitation, we propose a novel Structural Entropy-guided Knowledge Navigator (SENATOR) framework that addresses the intrinsic knowledge deficiencies of LLMs. Our approach employs the Structure Entropy (SE) metric to quantify uncertainty along knowledge graph paths and leverages Monte Carlo Tree Search (MCTS) to selectively explore regions where the model lacks domain-specific knowledge. Guided by these insights, the framework generates targeted synthetic data for supervised fine-tuning, enabling continuous self-improvement. Experimental results on LLaMA-3 and Qwen2 across multiple domain-specific benchmarks show that SENATOR effectively detects and repairs knowledge deficiencies, achieving notable performance improvements. The code and data for our methods and experiments are available at https://github.com/weiyifan1023/senator.