Ontology Completion with Natural Language Inference and Concept Embeddings: An Analysis

TL;DR

This paper compares natural language inference and concept embeddings for ontology completion, introduces a benchmark for evaluation, and finds that hybrid methods outperform individual approaches, though large language models still struggle.

Contribution

It provides a comprehensive analysis of two ontology completion approaches and introduces a new benchmark for their evaluation.

Findings

Hybrid strategies outperform individual methods.

Large Language Models face challenges in ontology completion.

The approaches are complementary, enhancing overall performance.

Abstract

We consider the problem of finding plausible knowledge that is missing from a given ontology, as a generalisation of the well-studied taxonomy expansion task. One line of work treats this task as a Natural Language Inference (NLI) problem, thus relying on the knowledge captured by language models to identify the missing knowledge. Another line of work uses concept embeddings to identify what different concepts have in common, taking inspiration from cognitive models for category based induction. These two approaches are intuitively complementary, but their effectiveness has not yet been compared. In this paper, we introduce a benchmark for evaluating ontology completion methods and thoroughly analyse the strengths and weaknesses of both approaches. We find that both approaches are indeed complementary, with hybrid strategies achieving the best overall results. We also find that the task is highly challenging for Large Language Models, even after fine-tuning.