PathISE: Learning Informative Path Supervision for Knowledge Graph Question Answering

TL;DR

PathISE introduces a lightweight framework that learns high-quality supervision signals for KGQA by estimating relation path informativeness, enabling improved reasoning without costly annotations.

Contribution

The paper presents PathISE, a novel method for generating pseudo supervision signals for KGQA using a transformer-based estimator, reducing reliance on expensive annotations.

Findings

PathISE achieves state-of-the-art results on three KGQA benchmarks.

It provides reusable supervision signals that enhance existing KGQA models.

The approach reduces the need for costly intermediate supervision signals.

Abstract

Knowledge Graph Question Answering (KGQA) aims to answer user questions by reasoning over Knowledge Graphs (KGs). Recent KGQA methods mainly follow the retrieval-augmented generation paradigm to ground Large Language Models~(LLMs) with structured knowledge from KGs. However, training effective models to retrieve question-relevant evidence from KGs typically requires high-quality intermediate supervision signals, such as question-relevant paths or subgraphs, which are time- and resource-intensive to obtain. We propose PathISE, a novel framework for learning high-quality intermediate supervision from answer-level labels. PathISE introduces a lightweight transformer-based estimator that estimates the informativeness of relation paths to construct pseudo path-level supervision. This supervision is then distilled into an LLM path generator, whose generated paths are grounded in the KG to provide compact evidence for inductive answer reasoning. ExtensiveISE experiments on three KGQA benchmarks show that PathISE achieves competitive or state-of-the-art KGQA performance, and provides reusable supervision signals that can enhance existing KGQA models, without relying on costly LLM-refined supervision signals. Our source code is available at https://anonymous.4open.science/r/PathISE-2F87.