GRainsaCK: a Comprehensive Software Library for Benchmarking Explanations of Link Prediction Tasks on Knowledge Graphs

TL;DR

GRainsaCK is a comprehensive, modular software library designed to standardize and streamline the benchmarking of explanation methods for link prediction in knowledge graphs, enhancing reproducibility and comparability.

Contribution

It introduces GRainsaCK, the first reusable software platform that standardizes the evaluation of explanation methods for link prediction in knowledge graphs.

Findings

Provides a unified evaluation protocol

Enhances modularity and extensibility

Includes extensive documentation and tutorials

Abstract

Since Knowledge Graphs are often incomplete, link prediction methods are adopted for predicting missing facts. Scalable embedding based solutions are mostly adopted for this purpose, however, they lack comprehensibility, which may be crucial in several domains. Explanation methods tackle this issue by identifying supporting knowledge explaining the predicted facts. Regretfully, evaluating/comparing quantitatively the resulting explanations is challenging as there is no standard evaluation protocol and overall benchmarking resource. We fill this important gap by proposing GRainsaCK, a reusable software resource that fully streamlines all the tasks involved in benchmarking explanations, i.e., from model training to evaluation of explanations along the same evaluation protocol. Moreover, GRainsaCK furthers modularity/extensibility by implementing the main components as functions that can be easily replaced. Finally, fostering its reuse, we provide extensive documentation including a tutorial.