TextBox 2.0: A Text Generation Library with Pre-trained Language Models

TL;DR

TextBox 2.0 is a comprehensive, user-friendly library that supports 13 text generation tasks, 83 datasets, 45 pre-trained models, and integrates the entire research pipeline for efficient text generation research.

Contribution

It introduces a unified, easy-to-use library for text generation that covers diverse tasks, datasets, models, and training strategies, streamlining research workflows.

Findings

Validated effectiveness through extensive experiments

Demonstrated support for diverse research scenarios

Showcased ease of use via Python API and command line

Abstract

To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2.0, focusing on the use of pre-trained language models (PLMs). To be comprehensive, our library covers $13$ common text generation tasks and their corresponding $83$ datasets and further incorporates $45$ PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLMs. We also implement $4$ efficient training strategies and provide $4$ generation objectives for pre-training new PLMs from scratch. To be unified, we design the interfaces to support the entire research pipeline (from data loading to training and evaluation), ensuring that each step can be fulfilled in a unified way. Despite the rich functionality, it is easy to use our library, either through the friendly Python API or command line. To validate the effectiveness of our library, we conduct extensive experiments and exemplify four types of research scenarios. The project is released at the link: https://github.com/RUCAIBox/TextBox.