Question Answering as Global Reasoning over Semantic Abstractions

TL;DR

This paper introduces SEMANTICILP, a novel reasoning system that leverages semantic abstractions and graph search to improve multiple-choice question answering, especially in data-scarce domains.

Contribution

It is the first system to reason over diverse semantic abstractions using off-the-shelf NLP modules for question answering.

Findings

Outperforms state-of-the-art models on science QA datasets.

Effective in domains with limited training data.

Generalizes prior structured QA approaches.

Abstract

We propose a novel method for exploiting the semantic structure of text to answer multiple-choice questions. The approach is especially suitable for domains that require reasoning over a diverse set of linguistic constructs but have limited training data. To address these challenges, we present the first system, to the best of our knowledge, that reasons over a wide range of semantic abstractions of the text, which are derived using off-the-shelf, general-purpose, pre-trained natural language modules such as semantic role labelers, coreference resolvers, and dependency parsers. Representing multiple abstractions as a family of graphs, we translate question answering (QA) into a search for an optimal subgraph that satisfies certain global and local properties. This formulation generalizes several prior structured QA systems. Our system, SEMANTICILP, demonstrates strong performance on two domains simultaneously. In particular, on a collection of challenging science QA datasets, it outperforms various state-of-the-art approaches, including neural models, broad coverage information retrieval, and specialized techniques using structured knowledge bases, by 2%-6%.