Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective

TL;DR

This paper introduces UAG, a framework that integrates uncertainty quantification into knowledge graph reasoning with LLMs, providing reliable predictions with theoretical guarantees and improved accuracy for high-stakes applications.

Contribution

The paper presents a novel uncertainty-aware reasoning framework for KG-LLMs that uses conformal prediction and error rate control to enhance reliability and reduce prediction uncertainty.

Findings

UAG achieves pre-defined coverage rates effectively.

Reduces prediction set size by 40% on average.

Provides theoretical guarantees on prediction reliability.

Abstract

Recently, Knowledge Graphs (KGs) have been successfully coupled with Large Language Models (LLMs) to mitigate their hallucinations and enhance their reasoning capability, such as in KG-based retrieval-augmented frameworks. However, current KG-LLM frameworks lack rigorous uncertainty estimation, limiting their reliable deployment in high-stakes applications. Directly incorporating uncertainty quantification into KG-LLM frameworks presents challenges due to their complex architectures and the intricate interactions between the knowledge graph and language model components. To address this gap, we propose a new trustworthy KG-LLM framework, Uncertainty Aware Knowledge-Graph Reasoning (UAG), which incorporates uncertainty quantification into the KG-LLM framework. We design an uncertainty-aware multi-step reasoning framework that leverages conformal prediction to provide a theoretical guarantee on the prediction set. To manage the error rate of the multi-step process, we additionally introduce an error rate control module to adjust the error rate within the individual components. Extensive experiments show that our proposed UAG can achieve any pre-defined coverage rate while reducing the prediction set/interval size by 40% on average over the baselines.