VANiLLa : Verbalized Answers in Natural Language at Large Scale

TL;DR

VANiLLa is a large-scale dataset of over 100,000 question-answer pairs in natural language designed to improve answer verbalization in knowledge graph question answering systems, facilitating machine learning approaches.

Contribution

The paper introduces the VANiLLa dataset, providing natural language answers for KGQA, addressing the lack of such datasets for training machine learning models.

Findings

Baseline models trained on VANiLLa show promising results.

The dataset enables research on answer verbalization in KGQA.

Answers are syntactically and semantically closer to questions.

Abstract

In the last years, there have been significant developments in the area of Question Answering over Knowledge Graphs (KGQA). Despite all the notable advancements, current KGQA datasets only provide the answers as the direct output result of the formal query, rather than full sentences incorporating question context. For achieving coherent answers sentence with the question's vocabulary, template-based verbalization so are usually employed for a better representation of answers, which in turn require extensive expert intervention. Thus, making way for machine learning approaches; however, there is a scarcity of datasets that empower machine learning models in this area. Hence, we provide the VANiLLa dataset which aims at reducing this gap by offering answers in natural language sentences. The answer sentences in this dataset are syntactically and semantically closer to the question than to the triple fact. Our dataset consists of over 100k simple questions adapted from the CSQA and SimpleQuestionsWikidata datasets and generated using a semi-automatic framework. We also present results of training our dataset on multiple baseline models adapted from current state-of-the-art Natural Language Generation (NLG) architectures. We believe that this dataset will allow researchers to focus on finding suitable methodologies and architectures for answer verbalization.