Jack the Reader - A Machine Reading Framework

TL;DR

Jack the Reader is a flexible framework designed to facilitate rapid prototyping, evaluation, and comparison of machine reading models across multiple NLP tasks like question answering and natural language inference.

Contribution

It introduces a unified, reusable framework that streamlines model development, evaluation, and dataset integration for various machine reading tasks.

Findings

Supports multiple tasks including QA, NLI, and Link Prediction

Enables quick model prototyping through component reuse

Facilitates evaluation on existing datasets and baseline models

Abstract

Many Machine Reading and Natural Language Understanding tasks require reading supporting text in order to answer questions. For example, in Question Answering, the supporting text can be newswire or Wikipedia articles; in Natural Language Inference, premises can be seen as the supporting text and hypotheses as questions. Providing a set of useful primitives operating in a single framework of related tasks would allow for expressive modelling, and easier model comparison and replication. To that end, we present Jack the Reader (Jack), a framework for Machine Reading that allows for quick model prototyping by component reuse, evaluation of new models on existing datasets as well as integrating new datasets and applying them on a growing set of implemented baseline models. Jack is currently supporting (but not limited to) three tasks: Question Answering, Natural Language Inference, and Link Prediction. It is developed with the aim of increasing research efficiency and code reuse.