Modern Baselines for SPARQL Semantic Parsing

TL;DR

This paper explores the use of pre-trained language models like T5, BART, and PGNs for generating SPARQL queries from natural language questions, achieving state-of-the-art results on key datasets and enabling new semantic parsing capabilities.

Contribution

It introduces PLM-based models as new strong baselines for SPARQL semantic parsing, demonstrating their effectiveness and novel copying capabilities.

Findings

T5 achieves state-of-the-art performance on LC-QuAD datasets.

PLMs outperform previous task-specific models.

The methods enable copying parts of input to output queries.

Abstract

In this work, we focus on the task of generating SPARQL queries from natural language questions, which can then be executed on Knowledge Graphs (KGs). We assume that gold entity and relations have been provided, and the remaining task is to arrange them in the right order along with SPARQL vocabulary, and input tokens to produce the correct SPARQL query. Pre-trained Language Models (PLMs) have not been explored in depth on this task so far, so we experiment with BART, T5 and PGNs (Pointer Generator Networks) with BERT embeddings, looking for new baselines in the PLM era for this task, on DBpedia and Wikidata KGs. We show that T5 requires special input tokenisation, but produces state of the art performance on LC-QuAD 1.0 and LC-QuAD 2.0 datasets, and outperforms task-specific models from previous works. Moreover, the methods enable semantic parsing for questions where a part of the input needs to be copied to the output query, thus enabling a new paradigm in KG semantic parsing.