Relational Graph Convolutional Neural Networks for Multihop Reasoning: A Comparative Study

TL;DR

This paper systematically evaluates different Relational Graph Convolutional Network architectures, relations, and embeddings to determine their impact on multihop question answering performance on the WikiHop dataset.

Contribution

It provides a comprehensive empirical analysis of RGCN components and their effects on multihop reasoning accuracy, filling a gap in understanding optimal configurations.

Findings

Certain relations significantly improve performance

Node embeddings influence reasoning accuracy

Model architecture choices impact results

Abstract

Multihop Question Answering is a complex Natural Language Processing task that requires multiple steps of reasoning to find the correct answer to a given question. Previous research has explored the use of models based on Graph Neural Networks for tackling this task. Various architectures have been proposed, including Relational Graph Convolutional Networks (RGCN). For these many node types and relations between them have been introduced, such as simple entity co-occurrences, modelling coreferences, or "reasoning paths" from questions to answers via intermediary entities. Nevertheless, a thoughtful analysis on which relations, node types, embeddings and architecture are the most beneficial for this task is still missing. In this paper we explore a number of RGCN-based Multihop QA models, graph relations, and node embeddings, and empirically explore the influence of each on Multihop QA performance on the WikiHop dataset.