GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer

TL;DR

GLiNER is a compact, bidirectional transformer-based NER model capable of recognizing any entity type, offering fast, parallel extraction and outperforming larger LLMs in zero-shot benchmarks.

Contribution

The paper introduces GLiNER, a novel, resource-efficient NER model that generalizes to any entity type and enables parallel extraction, surpassing existing LLMs in zero-shot performance.

Findings

GLiNER outperforms ChatGPT and fine-tuned LLMs in zero-shot NER tasks.

GLiNER enables fast, parallel entity extraction.

The model is effective across various NER benchmarks.

Abstract

Named Entity Recognition (NER) is essential in various Natural Language Processing (NLP) applications. Traditional NER models are effective but limited to a set of predefined entity types. In contrast, Large Language Models (LLMs) can extract arbitrary entities through natural language instructions, offering greater flexibility. However, their size and cost, particularly for those accessed via APIs like ChatGPT, make them impractical in resource-limited scenarios. In this paper, we introduce a compact NER model trained to identify any type of entity. Leveraging a bidirectional transformer encoder, our model, GLiNER, facilitates parallel entity extraction, an advantage over the slow sequential token generation of LLMs. Through comprehensive testing, GLiNER demonstrate strong performance, outperforming both ChatGPT and fine-tuned LLMs in zero-shot evaluations on various NER benchmarks.