TARexp: A Python Framework for Technology-Assisted Review Experiments

TL;DR

TARexp is an open source Python framework designed to simplify and standardize the experimentation process in technology-assisted review, enabling researchers to easily test and develop TAR algorithms.

Contribution

It introduces a flexible, declarative framework with state management for TAR experiments, integrating standard algorithms and supporting novel component development.

Findings

Facilitates reproducible TAR experiments

Supports complex workflow configurations

Enables integration of new TAR algorithms

Abstract

Technology-assisted review (TAR) is an important industrial application of information retrieval (IR) and machine learning (ML). While a small TAR research community exists, the complexity of TAR software and workflows is a major barrier to entry. Drawing on past open source TAR efforts, as well as design patterns from the IR and ML open source software, we present an open source Python framework for conducting experiments on TAR algorithms. Key characteristics of this framework are declarative representations of workflows and experiment plans, the ability for components to play variable numbers of workflow roles, and state maintenance and restart capabilities. Users can draw on reference implementations of standard TAR algorithms while incorporating novel components to explore their research interests. The framework is available at https://github.com/eugene-yang/tarexp.