Zshot: An Open-source Framework for Zero-Shot Named Entity Recognition and Relation Extraction

TL;DR

Zshot is an open-source framework designed to advance zero-shot named entity recognition and relation extraction by providing a comprehensive, extendible platform with benchmarking, APIs, and visualization tools for research and industry applications.

Contribution

It introduces Zshot, a novel framework that enables comparison of state-of-the-art ZSL methods, supports industry deployment, and includes enhancements like ensembling and visualization.

Findings

Supports benchmarking of ZSL methods on standard datasets

Provides APIs compatible with SpaCy for production use

Includes utilities for accuracy boosting and visualization

Abstract

The Zero-Shot Learning (ZSL) task pertains to the identification of entities or relations in texts that were not seen during training. ZSL has emerged as a critical research area due to the scarcity of labeled data in specific domains, and its applications have grown significantly in recent years. With the advent of large pretrained language models, several novel methods have been proposed, resulting in substantial improvements in ZSL performance. There is a growing demand, both in the research community and industry, for a comprehensive ZSL framework that facilitates the development and accessibility of the latest methods and pretrained models.In this study, we propose a novel ZSL framework called Zshot that aims to address the aforementioned challenges. Our primary objective is to provide a platform that allows researchers to compare different state-of-the-art ZSL methods with standard benchmark datasets. Additionally, we have designed our framework to support the industry with readily available APIs for production under the standard SpaCy NLP pipeline. Our API is extendible and evaluable, moreover, we include numerous enhancements such as boosting the accuracy with pipeline ensembling and visualization utilities available as a SpaCy extension.