SocraticKG: Knowledge Graph Construction via QA-Driven Fact Extraction

TL;DR

SocraticKG introduces a QA-driven method for knowledge graph construction that enhances factual coverage and relational coherence by using question-answer pairs as an intermediate step, improving reasoning and information retention.

Contribution

It presents a novel QA-based framework that systematically captures document semantics and implicit relations, addressing key limitations of existing LLM-based KG extraction methods.

Findings

Achieves better factual retention and structural cohesion in KGs.

Supports complex multi-hop reasoning tasks.

Outperforms existing methods on MINE and HotpotQA benchmarks.

Abstract

Constructing Knowledge Graphs (KGs) from unstructured text provides a structured framework for knowledge representation and reasoning, yet current LLM-based approaches struggle with a fundamental trade-off: factual coverage often leads to relational fragmentation, while premature consolidation causes information loss. To address this, we propose SocraticKG, an automated KG construction method that introduces question-answer pairs as a structured intermediate representation to systematically unfold document-level semantics prior to triple extraction. By employing 5W1H-guided QA expansion, SocraticKG captures contextual dependencies and implicit relational links typically lost in direct KG extraction pipelines, providing explicit grounding in the source document that helps mitigate implicit reasoning errors. Evaluation on the MINE benchmark and HotpotQA downstream task demonstrates that our approach effectively addresses the coverage-connectivity trade-off, achieving superior factual retention and structural cohesion while supporting complex multi-hop reasoning.