KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction
Jack Boylan, Shashank Mangla, Dominic Thorn, Demian Gholipour, Ghalandari, Parsa Ghaffari, Chris Hokamp

TL;DR
KGValidator leverages Large Language Models to automate the validation of knowledge graphs, reducing reliance on costly human annotation by using generative AI for structural, semantic, and factual verification.
Contribution
Introduces a flexible framework utilizing LLMs for automatic validation of knowledge graphs, integrating external knowledge sources and adaptable validation strategies.
Findings
Framework effectively automates KG validation tasks.
Supports referencing external knowledge for improved accuracy.
Easy to adapt for different types of graph-structured data.
Abstract
This study explores the use of Large Language Models (LLMs) for automatic evaluation of knowledge graph (KG) completion models. Historically, validating information in KGs has been a challenging task, requiring large-scale human annotation at prohibitive cost. With the emergence of general-purpose generative AI and LLMs, it is now plausible that human-in-the-loop validation could be replaced by a generative agent. We introduce a framework for consistency and validation when using generative models to validate knowledge graphs. Our framework is based upon recent open-source developments for structural and semantic validation of LLM outputs, and upon flexible approaches to fact checking and verification, supported by the capacity to reference external knowledge sources of any kind. The design is easy to adapt and extend, and can be used to verify any kind of graph-structured data through…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemantic Web and Ontologies · Advanced Graph Neural Networks · Data Mining Algorithms and Applications
