skweak: Weak Supervision Made Easy for NLP

TL;DR

skweak is an open-source Python toolkit that simplifies applying weak supervision in NLP, enabling automatic annotation of datasets through diverse labeling functions and aggregation models for tasks like NER and sentiment analysis.

Contribution

It introduces a versatile toolkit that streamlines the implementation and aggregation of weak supervision labeling functions specifically for NLP applications.

Findings

Facilitates easy implementation of diverse labeling functions

Supports unsupervised aggregation of weak labels

Applied successfully to NER and sentiment analysis tasks

Abstract

We present skweak, a versatile, Python-based software toolkit enabling NLP developers to apply weak supervision to a wide range of NLP tasks. Weak supervision is an emerging machine learning paradigm based on a simple idea: instead of labelling data points by hand, we use labelling functions derived from domain knowledge to automatically obtain annotations for a given dataset. The resulting labels are then aggregated with a generative model that estimates the accuracy (and possible confusions) of each labelling function. The skweak toolkit makes it easy to implement a large spectrum of labelling functions (such as heuristics, gazetteers, neural models or linguistic constraints) on text data, apply them on a corpus, and aggregate their results in a fully unsupervised fashion. skweak is especially designed to facilitate the use of weak supervision for NLP tasks such as text classification and sequence labelling. We illustrate the use of skweak for NER and sentiment analysis. skweak is released under an open-source license and is available at: https://github.com/NorskRegnesentral/skweak