Interactive Machine Comprehension with Information Seeking Agents

TL;DR

This paper introduces an interactive approach to machine reading comprehension that enables models to seek relevant information through sequential decision making, aiming to improve scalability for real-world web-level question answering.

Contribution

It reframes static MRC datasets as interactive, partially observable environments allowing models to actively seek information, thus enhancing scalability for practical QA applications.

Findings

Reframed MRC datasets as interactive environments

Developed a model that seeks relevant info via sequential decisions

Demonstrated potential for scaling to web-level QA

Abstract

Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA). We argue that this stems from the nature of MRC datasets: most of these are static environments wherein the supporting documents and all necessary information are fully observed. In this paper, we propose a simple method that reframes existing MRC datasets as interactive, partially observable environments. Specifically, we "occlude" the majority of a document's text and add context-sensitive commands that reveal "glimpses" of the hidden text to a model. We repurpose SQuAD and NewsQA as an initial case study, and then show how the interactive corpora can be used to train a model that seeks relevant information through sequential decision making. We believe that this setting can contribute in scaling models to web-level QA scenarios.