SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoning

TL;DR

SQUIRE is a novel sequence-to-sequence Transformer framework for multi-hop knowledge graph reasoning that improves accuracy and convergence speed, especially in sparse graphs, by generating reasoning paths without relying solely on existing edges.

Contribution

It introduces the first Transformer-based sequence-to-sequence model for multi-hop KG reasoning, enabling end-to-end learning and better handling of missing edges.

Findings

Achieves significant performance improvements over prior methods.

Converges 4 to 7 times faster than existing approaches.

Effectively infers missing edges in sparse knowledge graphs.

Abstract

Multi-hop knowledge graph (KG) reasoning has been widely studied in recent years to provide interpretable predictions on missing links with evidential paths. Most previous works use reinforcement learning (RL) based methods that learn to navigate the path towards the target entity. However, these methods suffer from slow and poor convergence, and they may fail to infer a certain path when there is a missing edge along the path. Here we present SQUIRE, the first Sequence-to-sequence based multi-hop reasoning framework, which utilizes an encoder-decoder Transformer structure to translate the query to a path. Our framework brings about two benefits: (1) It can learn and predict in an end-to-end fashion, which gives better and faster convergence; (2) Our Transformer model does not rely on existing edges to generate the path, and has the flexibility to complete missing edges along the path, especially in sparse KGs. Experiments on standard and sparse KGs show that our approach yields significant improvement over prior methods, while converging 4x-7x faster.