SPARKLE: Enhancing SPARQL Generation with Direct KG Integration in Decoding

TL;DR

SPARKLE is an end-to-end framework that improves natural language to SPARQL query generation by directly integrating knowledge base structure during decoding, reducing errors and adapting to evolving data.

Contribution

It introduces a novel decoding method that incorporates knowledge base structure directly, enhancing accuracy and adaptability over prior multi-stage and static models.

Findings

Achieves state-of-the-art results on SimpleQuestions-Wiki

Highest F1 score on LCQuAD 1.0 among non-gold entity models

Demonstrates fast inference and adaptability to knowledge base changes

Abstract

Existing KBQA methods have traditionally relied on multi-stage methodologies, involving tasks such as entity linking, subgraph retrieval and query structure generation. However, multi-stage approaches are dependent on the accuracy of preceding steps, leading to cascading errors and increased inference time. Although a few studies have explored the use of end-to-end models, they often suffer from lower accuracy and generate inoperative query that is not supported by the underlying data. Furthermore, most prior approaches are limited to the static training data, potentially overlooking the evolving nature of knowledge bases over time. To address these challenges, we present a novel end-to-end natural language to SPARQL framework, SPARKLE. Notably SPARKLE leverages the structure of knowledge base directly during the decoding, effectively integrating knowledge into the query generation. Our study reveals that simply referencing knowledge base during inference significantly reduces the occurrence of inexecutable query generations. SPARKLE achieves new state-of-the-art results on SimpleQuestions-Wiki and highest F1 score on LCQuAD 1.0 (among models not using gold entities), while getting slightly lower result on the WebQSP dataset. Finally, we demonstrate SPARKLE's fast inference speed and its ability to adapt when the knowledge base differs between the training and inference stages.