PyRelationAL: a python library for active learning research and development

TL;DR

PyRelationAL is an open-source Python library designed to facilitate active learning research by providing a modular, benchmarked, and extensible toolkit compatible with existing ML frameworks.

Contribution

It introduces a flexible, two-step design framework for composing and evaluating diverse active learning strategies within a unified programming model.

Findings

Includes a collection of benchmarks across multiple datasets.

Supports both single and batch acquisition strategies.

Facilitates comparison and development of new active learning methods.

Abstract

Active learning (AL) is a sub-field of ML focused on the development of methods to iteratively and economically acquire data by strategically querying new data points that are the most useful for a particular task. Here, we introduce PyRelationAL, an open source library for AL research. We describe a modular toolkit based around a two step design methodology for composing pool-based active learning strategies applicable to both single-acquisition and batch-acquisition strategies. This framework allows for the mathematical and practical specification of a broad number of existing and novel strategies under a consistent programming model and abstraction. Furthermore, we incorporate datasets and active learning tasks applicable to them to simplify comparative evaluation and benchmarking, along with an initial group of benchmarks across datasets included in this library. The toolkit is compatible with existing ML frameworks. PyRelationAL is maintained using modern software engineering practices -- with an inclusive contributor code of conduct -- to promote long term library quality and utilisation. PyRelationAL is available under a permissive Apache licence on PyPi and at https://github.com/RelationRx/pyrelational.