# Are Query-Based Ontology Debuggers Really Helping Knowledge Engineers?

## TL;DR

This paper evaluates the practical effectiveness of query-based ontology debugging through user studies, finding it more efficient than test case debugging but highlighting the importance of careful query design due to user errors.

## Contribution

It provides empirical evidence on the real-world usefulness of query-based ontology debugging, emphasizing user interaction challenges and advantages over algorithmic methods.

## Key findings

- Query-based debugging is more efficient than test case debugging.
- Users often make errors in the debugging process.
- Careful query design is crucial to reduce user errors.

## Abstract

Real-world semantic or knowledge-based systems, e.g., in the biomedical domain, can become large and complex. Tool support for the localization and repair of faults within knowledge bases of such systems can therefore be essential for their practical success. Correspondingly, a number of knowledge base debugging approaches, in particular for ontology-based systems, were proposed throughout recent years. Query-based debugging is a comparably recent interactive approach that localizes the true cause of an observed problem by asking knowledge engineers a series of questions. Concrete implementations of this approach exist, such as the OntoDebug plug-in for the ontology editor Prot\'eg\'e. To validate that a newly proposed method is favorable over an existing one, researchers often rely on simulation-based comparisons. Such an evaluation approach however has certain limitations and often cannot fully inform us about a method's true usefulness. We therefore conducted different user studies to assess the practical value of query-based ontology debugging. One main insight from the studies is that the considered interactive approach is indeed more efficient than an alternative algorithmic debugging based on test cases. We also observed that users frequently made errors in the process, which highlights the importance of a careful design of the queries that users need to answer.