A Comparative Study of Question Answering over Knowledge Bases

TL;DR

This paper compares six KBQA systems across multiple datasets, analyzes their strengths and weaknesses, and introduces an improved mapping algorithm and a multilingual COVID-19 knowledge base to advance the field.

Contribution

It provides a comprehensive comparison of KBQA systems, proposes an advanced mapping algorithm, and introduces a multilingual COVID-19 knowledge base for future research.

Findings

Existing KBQA systems vary in performance across question types and languages.

The proposed mapping algorithm improves accuracy of KBQA models.

The COVID-KGQA corpus supports multilingual and COVID-19 related question answering.

Abstract

Question answering over knowledge bases (KBQA) has become a popular approach to help users extract information from knowledge bases. Although several systems exist, choosing one suitable for a particular application scenario is difficult. In this article, we provide a comparative study of six representative KBQA systems on eight benchmark datasets. In that, we study various question types, properties, languages, and domains to provide insights on where existing systems struggle. On top of that, we propose an advanced mapping algorithm to aid existing models in achieving superior results. Moreover, we also develop a multilingual corpus COVID-KGQA, which encourages COVID-19 research and multilingualism for the diversity of future AI. Finally, we discuss the key findings and their implications as well as performance guidelines and some future improvements. Our source code is available at \url{https://github.com/tamlhp/kbqa}.