Improving Multi-hop Logical Reasoning in Knowledge Graphs with Context-Aware Query Representation Learning

TL;DR

This paper introduces a context-aware query representation learning method that improves multi-hop logical reasoning on knowledge graphs by integrating structural and relation-induced contexts, leading to significant performance gains.

Contribution

It proposes a novel dual-context paradigm that enhances existing reasoning models by incorporating query structure and relation information, addressing limitations of linear sequential approaches.

Findings

Achieves up to 19.5% performance improvement on two datasets.

Effectively integrates query structure and relation context into reasoning models.

Enhances the internal representations of nodes during multi-hop reasoning.

Abstract

Multi-hop logical reasoning on knowledge graphs is a pivotal task in natural language processing, with numerous approaches aiming to answer First-Order Logic (FOL) queries. Recent geometry (e.g., box, cone) and probability (e.g., beta distribution)-based methodologies have effectively addressed complex FOL queries. However, a common challenge across these methods lies in determining accurate geometric bounds or probability parameters for these queries. The challenge arises because existing methods rely on linear sequential operations within their computation graphs, overlooking the logical structure of the query and the relation-induced information that can be gleaned from the relations of the query, which we call the context of the query. To address the problem, we propose a model-agnostic methodology that enhances the effectiveness of existing multi-hop logical reasoning approaches by fully integrating the context of the FOL query graph. Our approach distinctively discerns (1) the structural context inherent to the query structure and (2) the relation-induced context unique to each node in the query graph as delineated in the corresponding knowledge graph. This dual-context paradigm helps nodes within a query graph attain refined internal representations throughout the multi-hop reasoning steps. Through experiments on two datasets, our method consistently enhances the three multi-hop reasoning foundation models, achieving performance improvements of up to 19.5%. Our code is available at https://github.com/kjh9503/caqr.