KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models
Haris Widjaja, Kiril Gashteovski, Wiem Ben Rim, Pengfei Liu,, Christopher Malon, Daniel Ruffinelli, Carolin Lawrence, Graham Neubig

TL;DR
KGxBoard is an interactive framework that enables detailed, interpretable evaluation of knowledge graph completion models by analyzing their performance on specific data subsets, revealing insights beyond average metrics.
Contribution
It introduces KGxBoard, a novel tool for fine-grained, explainable evaluation of KGC models on meaningful data subsets, addressing limitations of traditional single-score metrics.
Findings
Revealed model strengths and weaknesses on specific data aspects
Identified capabilities that are not apparent with average metrics
Enhanced understanding of model behavior and learning patterns
Abstract
Knowledge Graphs (KGs) store information in the form of (head, predicate, tail)-triples. To augment KGs with new knowledge, researchers proposed models for KG Completion (KGC) tasks such as link prediction; i.e., answering (h; p; ?) or (?; p; t) queries. Such models are usually evaluated with averaged metrics on a held-out test set. While useful for tracking progress, averaged single-score metrics cannot reveal what exactly a model has learned -- or failed to learn. To address this issue, we propose KGxBoard: an interactive framework for performing fine-grained evaluation on meaningful subsets of the data, each of which tests individual and interpretable capabilities of a KGC model. In our experiments, we highlight the findings that we discovered with the use of KGxBoard, which would have been impossible to detect with standard averaged single-score metrics.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Bayesian Modeling and Causal Inference · Topic Modeling
MethodsTest
