MedRule-KG: A Knowledge-Graph--Steered Scaffold for Reliable Mathematical and Biomedical Reasoning

TL;DR

MedRule-KG integrates a knowledge-graph scaffold and verifier into large language models to enhance the reliability of scientific reasoning and biomedical tasks, significantly reducing errors and violations.

Contribution

This work introduces MedRule-KG, a novel framework combining knowledge graphs and deterministic verification to improve the validity of LLM outputs in scientific and biomedical domains.

Findings

Reduces violation counts by 83.2% compared to chain-of-thought baseline.

Improves exact match scores across 90 diverse tasks.

Verifier adds negligible latency, enabling practical interactive use.

Abstract

We study how to impose domain-consistent structure on large language models (LLMs) used for scientific reasoning and early-stage drug discovery. We present MedRule-KG, a compact knowledge-graph scaffold paired with a lightweight verifier that steers generation toward mathematically and biomedically valid outputs. The system injects curated symbolic facts into prompts and then enforces rule satisfaction with a deterministic checker. We formalize generation as constrained inference, introduce a soft guidance surrogate suitable for decoding, and perform a thorough statistical analysis with uncertainty quantification. Across 90 tasks spanning reaction feasibility, metabolic compatibility, and toxicity screening, MedRule-KG reduces violation counts by 83.2\% relative to a strong chain-of-thought baseline while improving exact match. Results remain stable under stratification and scale with dataset size, and the verifier adds negligible latency, making the approach practical for interactive design.