OntoRAG: Enhancing Question-Answering through Automated Ontology Derivation from Unstructured Knowledge Bases

TL;DR

OntoRAG is an automated pipeline that derives ontologies from unstructured knowledge bases, significantly improving question-answering systems by transforming raw data into structured, queryable knowledge representations.

Contribution

This work introduces OntoRAG, a novel automated method for ontology creation from unstructured data, reducing manual effort and enhancing QA system capabilities.

Findings

Achieved 85% comprehensiveness win rate over vector RAG.

Outperformed GraphRAG in diversity and coverage.

Demonstrated effectiveness on electrical relay documents.

Abstract

Ontologies are pivotal for structuring knowledge bases to enhance question answering (QA) systems powered by Large Language Models (LLMs). However, traditional ontology creation relies on manual efforts by domain experts, a process that is time intensive, error prone, and impractical for large, dynamic knowledge domains. This paper introduces OntoRAG, an automated pipeline designed to derive ontologies from unstructured knowledge bases, with a focus on electrical relay documents. OntoRAG integrates advanced techniques, including web scraping, PDF parsing, hybrid chunking, information extraction, knowledge graph construction, and ontology creation, to transform unstructured data into a queryable ontology. By leveraging LLMs and graph based methods, OntoRAG enhances global sensemaking capabilities, outperforming conventional Retrieval Augmented Generation (RAG) and GraphRAG approaches in comprehensiveness and diversity. Experimental results demonstrate OntoRAGs effectiveness, achieving a comprehensiveness win rate of 85% against vector RAG and 75% against GraphRAGs best configuration. This work addresses the critical challenge of automating ontology creation, advancing the vision of the semantic web.