KGQuest: Template-Driven QA Generation from Knowledge Graphs with LLM-Based Refinement

TL;DR

This paper introduces a scalable, template-driven method for generating high-quality, factually accurate question-answer pairs from knowledge graphs, enhanced by LLM-based refinement for improved linguistic quality.

Contribution

It proposes a deterministic pipeline that clusters KG triplets into reusable templates and refines them with LLMs, addressing scalability and linguistic quality issues in QA generation from KGs.

Findings

Efficient generation of high-quality QA pairs from KGs.

Enhanced linguistic quality through LLM-based template refinement.

Maintains factual accuracy while improving fluency.

Abstract

The generation of questions and answers (QA) from knowledge graphs (KG) plays a crucial role in the development and testing of educational platforms, dissemination tools, and large language models (LLM). However, existing approaches often struggle with scalability, linguistic quality, and factual consistency. This paper presents a scalable and deterministic pipeline for generating natural language QA from KGs, with an additional refinement step using LLMs to further enhance linguistic quality. The approach first clusters KG triplets based on their relations, creating reusable templates through natural language rules derived from the entity types of objects and relations. A module then leverages LLMs to refine these templates, improving clarity and coherence while preserving factual accuracy. Finally, the instantiation of answer options is achieved through a selection strategy that introduces distractors from the KG. Our experiments demonstrate that this hybrid approach efficiently generates high-quality QA pairs, combining scalability with fluency and linguistic precision.