Incorporating Connections Beyond Knowledge Embeddings: A Plug-and-Play Module to Enhance Commonsense Reasoning in Machine Reading Comprehension

TL;DR

This paper introduces PIECER, a plug-and-play module that enhances commonsense reasoning in machine reading comprehension by explicitly utilizing knowledge graph connections, improving performance especially in low-resource scenarios.

Contribution

The paper proposes a novel, generalizable module that incorporates knowledge graph connections into MRC models to improve commonsense reasoning capabilities.

Findings

PIECER improves performance on ReCoRD dataset

Enhances low-resource MRC model accuracy

Applicable to various base models

Abstract

Conventional Machine Reading Comprehension (MRC) has been well-addressed by pattern matching, but the ability of commonsense reasoning remains a gap between humans and machines. Previous methods tackle this problem by enriching word representations via pre-trained Knowledge Graph Embeddings (KGE). However, they make limited use of a large number of connections between nodes in Knowledge Graphs (KG), which could be pivotal cues to build the commonsense reasoning chains. In this paper, we propose a Plug-and-play module to IncorporatE Connection information for commonsEnse Reasoning (PIECER). Beyond enriching word representations with knowledge embeddings, PIECER constructs a joint query-passage graph to explicitly guide commonsense reasoning by the knowledge-oriented connections between words. Further, PIECER has high generalizability since it can be plugged into suitable positions in any MRC model. Experimental results on ReCoRD, a large-scale public MRC dataset requiring commonsense reasoning, show that PIECER introduces stable performance improvements for four representative base MRC models, especially in low-resource settings.