jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models

TL;DR

Jiant is an open source toolkit designed for multitask and transfer learning experiments on English natural language understanding tasks, supporting over 50 tasks including GLUE and SuperGLUE benchmarks.

Contribution

It provides a modular, configurable platform for probing, transfer learning, and multitask training with state-of-the-art models like BERT and RoBERTa.

Findings

Successfully reproduces published performance on multiple NLU tasks.

Supports a broad set of tasks including all GLUE and SuperGLUE benchmarks.

Facilitates research in general-purpose text understanding models.

Abstract

We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. jiant enables modular and configuration-driven experimentation with state-of-the-art models and implements a broad set of tasks for probing, transfer learning, and multitask training experiments. jiant implements over 50 NLU tasks, including all GLUE and SuperGLUE benchmark tasks. We demonstrate that jiant reproduces published performance on a variety of tasks and models, including BERT and RoBERTa. jiant is available at https://jiant.info.