Differentiable Reasoning over a Virtual Knowledge Base

TL;DR

This paper introduces DrKIT, a differentiable neural module that performs multi-hop reasoning over textual data treated as a virtual knowledge base, significantly improving accuracy and efficiency in complex question answering tasks.

Contribution

The paper presents DrKIT, a novel differentiable reasoning module that traverses textual data as a virtual KB, enabling end-to-end training and improved multi-hop question answering performance.

Findings

DrKIT improves accuracy by 9 points on 3-hop questions in MetaQA.

DrKIT reduces the gap between text-based and KB-based state-of-the-art by 70%.

DrKIT processes 10-100x more queries per second than existing systems.

Abstract

We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB). In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus. At each step the module uses a combination of sparse-matrix TFIDF indices and a maximum inner product search (MIPS) on a special index of contextual representations of the mentions. This module is differentiable, so the full system can be trained end-to-end using gradient based methods, starting from natural language inputs. We also describe a pretraining scheme for the contextual representation encoder by generating hard negative examples using existing knowledge bases. We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%. On HotpotQA, DrKIT leads to a 10% improvement over a BERT-based re-ranking approach to retrieving the relevant passages required to answer a question. DrKIT is also very efficient, processing 10-100x more queries per second than existing multi-hop systems.