From Prompt to Graph: Comparing LLM-Based Information Extraction Strategies in Domain-Specific Ontology Development

TL;DR

This paper compares three LLM-based strategies for automating ontology development in specialized domains, demonstrating their effectiveness in extracting domain-specific terms and relations with limited data.

Contribution

It introduces and evaluates three novel LLM-based methods for ontology construction, highlighting their advantages over traditional manual approaches.

Findings

Pre-trained LLMs effectively extract domain terms.

In-context learning performs well with limited data.

Fine-tuning improves extraction accuracy.

Abstract

Ontologies are essential for structuring domain knowledge, improving accessibility, sharing, and reuse. However, traditional ontology construction relies on manual annotation and conventional natural language processing (NLP) techniques, making the process labour-intensive and costly, especially in specialised fields like casting manufacturing. The rise of Large Language Models (LLMs) offers new possibilities for automating knowledge extraction. This study investigates three LLM-based approaches, including pre-trained LLM-driven method, in-context learning (ICL) method and fine-tuning method to extract terms and relations from domain-specific texts using limited data. We compare their performances and use the best-performing method to build a casting ontology that validated by domian expert.