Towards Automated Ontology Generation from Unstructured Text: A Multi-Agent LLM Approach

TL;DR

This paper investigates how multi-agent LLM architectures improve automated ontology generation from unstructured text, emphasizing planning and artifact roles to enhance quality and usability.

Contribution

It introduces a multi-agent architecture with specialized roles that significantly improves ontology quality and queryability over single-agent baselines.

Findings

Multi-agent approach improves structural quality of ontologies.

Planning-first strategy enhances ontology queryability.

Artifact-driven roles lead to more auditable ontology generation.

Abstract

Automatically generating formal ontologies from unstructured natural language remains a central challenge in knowledge engineering. While large language models (LLMs) show promise, it remains unclear which architectural design choices drive generation quality and why current approaches fail. We present a controlled experimental study using domain-specific insurance contracts to investigate these questions. We first establish a single-agent LLM baseline, identifying key failure modes such as poor Ontology Design Pattern compliance, structural redundancy, and ineffective iterative repair. We then introduce a multi-agent architecture that decomposes ontology construction into four artifact-driven roles: Domain Expert, Manager, Coder, and Quality Assurer. We evaluate performance across architectural quality (via a panel of heterogeneous LLM judges) and functional usability (via competency question driven SPARQL evaluation with complementary retrieval augmented generation based assessment). Results show that the multi-agent approach significantly improves structural quality and modestly enhances queryability, with gains driven primarily by front-loaded planning. These findings highlight planning-first, artifact-driven generation as a promising and more auditable path toward scalable automated ontology engineering.