TOKEN is a MASK: Few-shot Named Entity Recognition with Pre-trained Language Models

TL;DR

This paper introduces a simple, versatile few-shot and zero-shot domain adaptation method for Named Entity Recognition that leverages pre-trained language models with descriptive patterns, improving performance over baselines.

Contribution

The paper proposes a novel two-step approach combining a variable base module and a template module to enhance NER in low-data scenarios using pre-trained models.

Findings

Boosts state-of-the-art F1-score by 2-5% in various datasets.

Effective in both few-shot and zero-shot NER tasks.

Utilizes simple descriptive patterns to leverage pre-trained language models.

Abstract

Transferring knowledge from one domain to another is of practical importance for many tasks in natural language processing, especially when the amount of available data in the target domain is limited. In this work, we propose a novel few-shot approach to domain adaptation in the context of Named Entity Recognition (NER). We propose a two-step approach consisting of a variable base module and a template module that leverages the knowledge captured in pre-trained language models with the help of simple descriptive patterns. Our approach is simple yet versatile and can be applied in few-shot and zero-shot settings. Evaluating our lightweight approach across a number of different datasets shows that it can boost the performance of state-of-the-art baselines by 2-5% F1-score.