Complex Logical Query Answering by Calibrating Knowledge Graph Completion Models

TL;DR

This paper introduces CKGC, a lightweight calibration method that improves knowledge graph completion models for complex logical query answering by aligning prediction scores with true/false facts, boosting performance on benchmarks.

Contribution

The paper proposes a novel calibration technique, CKGC, that enhances KGC models' ability to answer complex logical queries without sacrificing ranking metrics.

Findings

Significant performance improvements on three benchmark datasets.

Calibration aligns prediction scores with true/false facts effectively.

Method preserves original ranking evaluation metrics.

Abstract

Complex logical query answering (CLQA) is a challenging task that involves finding answer entities for complex logical queries over incomplete knowledge graphs (KGs). Previous research has explored the use of pre-trained knowledge graph completion (KGC) models, which can predict the missing facts in KGs, to answer complex logical queries. However, KGC models are typically evaluated using ranking evaluation metrics, which may result in values of predictions of KGC models that are not well-calibrated. In this paper, we propose a method for calibrating KGC models, namely CKGC, which enables KGC models to adapt to answering complex logical queries. Notably, CKGC is lightweight and effective. The adaptation function is simple, allowing the model to quickly converge during the adaptation process. The core concept of CKGC is to map the values of predictions of KGC models to the range [0, 1], ensuring that values associated with true facts are close to 1, while values linked to false facts are close to 0. Through experiments on three benchmark datasets, we demonstrate that our proposed calibration method can significantly boost model performance in the CLQA task. Moreover, our approach can enhance the performance of CLQA while preserving the ranking evaluation metrics of KGC models. The code is available at https://github.com/changyi7231/CKGC.