Ontology Learning and Knowledge Graph Construction: A Comparison of Approaches and Their Impact on RAG Performance

TL;DR

This paper compares various knowledge graph construction methods for Retrieval-Augmented Generation systems, showing that ontology-guided KGs from relational databases are cost-effective and perform competitively with text-derived ontologies.

Contribution

It provides a comprehensive comparison of KG construction strategies and demonstrates the effectiveness of ontology-guided KGs in RAG performance, highlighting cost and complexity benefits.

Findings

Ontology-guided KGs with chunk info outperform vector baselines.

Relational database-based KGs perform comparably to text-based ones.

Database-based KGs reduce ontology learning costs and complexity.

Abstract

Retrieval-Augmented Generation (RAG) systems combine Large Language Models (LLMs) with external knowledge, and their performance depends heavily on how that knowledge is represented. This study investigates how different Knowledge Graph (KG) construction strategies influence RAG performance. We compare a variety of approaches: standard vector-based RAG, GraphRAG, and retrieval over KGs built from ontologies derived either from relational databases or textual corpora. Results show that ontology-guided KGs incorporating chunk information achieve competitive performance with state-of-the-art frameworks, substantially outperforming vector retrieval baselines. Moreover, the findings reveal that ontology-guided KGs built from relational databases perform competitively to ones built with ontologies extracted from text, with the benefit of offering a dual advantage: they require a one-time-only ontology learning process, substantially reducing LLM usage costs; and avoid the complexity of ontology merging inherent to text-based approaches.