Evontree: Ontology Rule-Guided Self-Evolution of Large Language Models

TL;DR

Evontree is a novel ontology-guided self-evolution method that enhances large language models in low-resource, knowledge-intensive domains like healthcare by detecting and reinforcing domain knowledge gaps.

Contribution

This paper introduces Evontree, a new ontology rule-guided approach for self-evolving LLMs in specialized domains with limited data, addressing knowledge inconsistency and improving accuracy.

Findings

Outperforms baseline models with up to 3.7% accuracy improvement.

Effectively detects and reinforces domain knowledge gaps.

Proves robustness through extensive ablation studies.

Abstract

Although Large Language Models (LLMs) perform exceptionally well in general domains, the problem of hallucinations poses significant risks in specialized fields such as healthcare and law, where high interpretability is essential. Existing fine-tuning methods depend heavily on large-scale professional datasets, which are often hard to obtain due to the privacy regulations. Moreover, existing self-evolution methods are primarily designed for general domains, which may struggle to adapt to knowledge-intensive domains due to the lack of knowledge constraints. In this paper, we propose an ontology rule guided method Evontree to enable self-evolution of LLMs in low-resource specialized domains. Specifically, Evontree first extracts domain ontology knowledge from raw models, then detects knowledge inconsistencies using two core ontology rules, and finally reinforces gap knowledge into model via self-distilled fine-tuning. Extensive evaluations on medical QA benchmarks using Llama3-8B-Instruct and Med42-V2 demonstrate the effectiveness of Evontree, which outperforms both the base models and strong baselines, achieving up to a 3.7\% improvement in accuracy. Detailed ablation studies further validate the robustness of our approach.