Self-Improvement Programming for Temporal Knowledge Graph Question Answering

TL;DR

This paper introduces Prog-TQA, a novel method that uses large language models with self-improvement strategies to explicitly model complex temporal constraints in question answering over temporal knowledge graphs, outperforming existing methods.

Contribution

It proposes a self-improvement programming approach leveraging LLMs for explicit modeling of temporal constraints in TKGQA, which is more comprehensive than prior implicit embedding methods.

Findings

Achieves superior performance on MultiTQ and CronQuestions datasets.

Demonstrates effectiveness of self-improvement strategy for LLMs in TKGQA.

Outperforms existing end-to-end methods in Hits@1 metric.

Abstract

Temporal Knowledge Graph Question Answering (TKGQA) aims to answer questions with temporal intent over Temporal Knowledge Graphs (TKGs). The core challenge of this task lies in understanding the complex semantic information regarding multiple types of time constraints (e.g., before, first) in questions. Existing end-to-end methods implicitly model the time constraints by learning time-aware embeddings of questions and candidate answers, which is far from understanding the question comprehensively. Motivated by semantic-parsing-based approaches that explicitly model constraints in questions by generating logical forms with symbolic operators, we design fundamental temporal operators for time constraints and introduce a novel self-improvement Programming method for TKGQA (Prog-TQA). Specifically, Prog-TQA leverages the in-context learning ability of Large Language Models (LLMs) to understand the combinatory time constraints in the questions and generate corresponding program drafts with a few examples given. Then, it aligns these drafts to TKGs with the linking module and subsequently executes them to generate the answers. To enhance the ability to understand questions, Prog-TQA is further equipped with a self-improvement strategy to effectively bootstrap LLMs using high-quality self-generated drafts. Extensive experiments demonstrate the superiority of the proposed Prog-TQA on MultiTQ and CronQuestions datasets, especially in the Hits@1 metric.