Benchmarking Knowledge Graphs on the Web

TL;DR

This paper reviews existing benchmarks for evaluating Knowledge Graph systems, emphasizing the importance of standardized comparisons to advance explainable AI development.

Contribution

It provides a comprehensive overview of current benchmarking methods for Knowledge Graphs, highlighting their role in enabling measurable progress.

Findings

Overview of existing Knowledge Graph benchmarks

Highlights the importance of standardized evaluation

Encourages development of new benchmarking methods

Abstract

The growing interest in making use of Knowledge Graphs for developing explainable artificial intelligence, there is an increasing need for a comparable and repeatable comparison of the performance of Knowledge Graph-based systems. History in computer science has shown that a main driver to scientific advances, and in fact a core element of the scientific method as a whole, is the provision of benchmarks to make progress measurable. This paper gives an overview of benchmarks used to evaluate systems that process Knowledge Graphs.