Credible Intervals for Knowledge Graph Accuracy Estimation

TL;DR

This paper introduces Bayesian Credible Intervals and an adaptive algorithm for more reliable and efficient accuracy estimation of large-scale Knowledge Graphs, improving upon traditional confidence interval methods.

Contribution

It proposes the use of Credible Intervals based on Bayesian statistics for KG accuracy estimation and introduces a new adaptive algorithm, aHPD, for improved efficiency and reliability.

Findings

CrIs provide stronger guarantees than CIs for KG accuracy.

The aHPD algorithm enhances efficiency in large-scale KG evaluation.

CrIs improve interpretability and reliability of accuracy estimates.

Abstract

Knowledge Graphs (KGs) are widely used in data-driven applications and downstream tasks, such as virtual assistants, recommendation systems, and semantic search. The accuracy of KGs directly impacts the reliability of the inferred knowledge and outcomes. Therefore, assessing the accuracy of a KG is essential for ensuring the quality of facts used in these tasks. However, the large size of real-world KGs makes manual triple-by-triple annotation impractical, thereby requiring sampling strategies to provide accuracy estimates with statistical guarantees. The current state-of-the-art approaches rely on Confidence Intervals (CIs), derived from frequentist statistics. While efficient, CIs have notable limitations and can lead to interpretation fallacies. In this paper, we propose to overcome the limitations of CIs by using \emph{Credible Intervals} (CrIs), which are grounded in Bayesian statistics. These intervals are more suitable for reliable post-data inference, particularly in KG accuracy evaluation. We prove that CrIs offer greater reliability and stronger guarantees than frequentist approaches in this context. Additionally, we introduce \emph{a}HPD, an adaptive algorithm that is more efficient for real-world KGs and statistically robust, addressing the interpretive challenges of CIs.