Question Answering on Scholarly Knowledge Graphs

TL;DR

This paper introduces JarvisQA, a BERT-based system designed to answer questions on scholarly knowledge graphs represented as tables, improving retrieval accuracy over existing methods and providing a reusable benchmark dataset.

Contribution

The paper presents JarvisQA, a novel BERT-based question answering system for scholarly tables, along with a new benchmark dataset for evaluating such systems.

Findings

JarvisQA outperforms baselines by 2-3 times in accuracy.

A new dataset of scholarly tables and questions is introduced.

The system effectively retrieves answers from diverse table formats.

Abstract

Answering questions on scholarly knowledge comprising text and other artifacts is a vital part of any research life cycle. Querying scholarly knowledge and retrieving suitable answers is currently hardly possible due to the following primary reason: machine inactionable, ambiguous and unstructured content in publications. We present JarvisQA, a BERT based system to answer questions on tabular views of scholarly knowledge graphs. Such tables can be found in a variety of shapes in the scholarly literature (e.g., surveys, comparisons or results). Our system can retrieve direct answers to a variety of different questions asked on tabular data in articles. Furthermore, we present a preliminary dataset of related tables and a corresponding set of natural language questions. This dataset is used as a benchmark for our system and can be reused by others. Additionally, JarvisQA is evaluated on two datasets against other baselines and shows an improvement of two to three folds in performance compared to related methods.