Template-free Prompt Tuning for Few-shot NER

TL;DR

This paper introduces a template-free prompt tuning approach for few-shot Named Entity Recognition (NER) that reformulates the task as a language modeling problem, eliminating template design and significantly speeding up decoding.

Contribution

The work proposes a novel template-free method for few-shot NER that simplifies the process and improves speed by predicting label words directly without templates.

Findings

Outperforms template-based methods in few-shot NER accuracy

Decoding speed is up to 1930 times faster than template-based approaches

Effective automatic search for appropriate label words enhances model adaptation

Abstract

Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words. However, when applied to token-level labeling tasks such as NER, it would be time-consuming to enumerate the template queries over all potential entity spans. In this work, we propose a more elegant method to reformulate NER tasks as LM problems without any templates. Specifically, we discard the template construction process while maintaining the word prediction paradigm of pre-training models to predict a class-related pivot word (or label word) at the entity position. Meanwhile, we also explore principled ways to automatically search for appropriate label words that the pre-trained models can easily adapt to. While avoiding complicated template-based process, the proposed LM objective also reduces the gap between different objectives used in pre-training and fine-tuning, thus it can better benefit the few-shot performance. Experimental results demonstrate the effectiveness of the proposed method over bert-tagger and template-based method under few-shot setting. Moreover, the decoding speed of the proposed method is up to 1930.12 times faster than the template-based method.