Explicit Contextual Semantics for Text Comprehension

TL;DR

This paper introduces a method to improve text comprehension by explicitly incorporating semantic role labels into deep learning models, significantly enhancing performance on benchmark reading comprehension and inference tasks.

Contribution

It is the first work to formally integrate semantic role labeling with text comprehension models to improve understanding and inference capabilities.

Findings

Achieved state-of-the-art results on benchmark datasets.

Explicit semantic role labels improve model performance.

Enhancement is compatible with existing pretrained language models.

Abstract

Who did what to whom is a major focus in natural language understanding, which is right the aim of semantic role labeling (SRL) task. Despite of sharing a lot of processing characteristics and even task purpose, it is surprisingly that jointly considering these two related tasks was never formally reported in previous work. Thus this paper makes the first attempt to let SRL enhance text comprehension and inference through specifying verbal predicates and their corresponding semantic roles. In terms of deep learning models, our embeddings are enhanced by explicit contextual semantic role labels for more fine-grained semantics. We show that the salient labels can be conveniently added to existing models and significantly improve deep learning models in challenging text comprehension tasks. Extensive experiments on benchmark machine reading comprehension and inference datasets verify that the proposed semantic learning helps our system reach new state-of-the-art over strong baselines which have been enhanced by well pretrained language models from the latest progress.