LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting

TL;DR

LightNER introduces a lightweight, pluggable prompting approach for low-resource NER that effectively addresses class and domain transfer issues without extensive retraining.

Contribution

The paper proposes a novel lightweight tuning paradigm with a unified verbalizer and guidance module, enabling effective low-resource NER with minimal parameter updates.

Findings

Achieves comparable performance to supervised methods in standard settings.

Outperforms strong baselines in low-resource scenarios.

Maintains model flexibility and transferability across domains.

Abstract

Most NER methods rely on extensive labeled data for model training, which struggles in the low-resource scenarios with limited training data. Existing dominant approaches usually suffer from the challenge that the target domain has different label sets compared with a resource-rich source domain, which can be concluded as class transfer and domain transfer. In this paper, we propose a lightweight tuning paradigm for low-resource NER via pluggable prompting (LightNER). Specifically, we construct the unified learnable verbalizer of entity categories to generate the entity span sequence and entity categories without any label-specific classifiers, thus addressing the class transfer issue. We further propose a pluggable guidance module by incorporating learnable parameters into the self-attention layer as guidance, which can re-modulate the attention and adapt pre-trained weights. Note that we only tune those inserted module with the whole parameter of the pre-trained language model fixed, thus, making our approach lightweight and flexible for low-resource scenarios and can better transfer knowledge across domains. Experimental results show that LightNER can obtain comparable performance in the standard supervised setting and outperform strong baselines in low-resource settings. Code is in https://github.com/zjunlp/DeepKE/tree/main/example/ner/few-shot.