My Ontologist: Evaluating BFO-Based AI for Definition Support

TL;DR

This paper evaluates GPT-4's ability to support BFO-based ontology development, introduces a specialized model 'My Ontologist', and discusses challenges in maintaining standard conformity amidst AI evolution.

Contribution

It presents the development and assessment of 'My Ontologist', a GPT-based tool designed to generate BFO-conformant ontologies, highlighting the challenges of aligning AI outputs with top-level standards.

Findings

GPT-4 can support BFO ontology generation with structured rules.

Model behavior changes impact ontology quality.

Alignment with top-level standards is crucial for AI-assisted ontology engineering.

Abstract

Generative artificial intelligence (AI), exemplified by the release of GPT-3.5 in 2022, has significantly advanced the potential applications of large language models (LLMs), including in the realms of ontology development and knowledge graph creation. Ontologies, which are structured frameworks for organizing information, and knowledge graphs, which combine ontologies with actual data, are essential for enabling interoperability and automated reasoning. However, current research has largely overlooked the generation of ontologies extending from established upper-level frameworks like the Basic Formal Ontology (BFO), risking the creation of non-integrable ontology silos. This study explores the extent to which LLMs, particularly GPT-4, can support ontologists trained in BFO. Through iterative development of a specialized GPT model named "My Ontologist," we aimed to generate BFO-conformant ontologies. Initial versions faced challenges in maintaining definition conventions and leveraging foundational texts effectively. My Ontologist 3.0 showed promise by adhering to structured rules and modular ontology suites, yet the release of GPT-4o disrupted this progress by altering the model's behavior. Our findings underscore the importance of aligning LLM-generated ontologies with top-level standards and highlight the complexities of integrating evolving AI capabilities in ontology engineering.