Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER

TL;DR

This paper introduces a simple demonstration-based learning approach for low-resource named entity recognition, improving performance by carefully selecting task demonstrations and templates, reducing the need for extensive labeled data.

Contribution

It proposes a demonstration-based learning method for NER that systematically studies demonstration strategies, showing significant improvements in low-resource and domain adaptation scenarios.

Findings

Performance improves by 4-17% with demonstration strategies

Good demonstrations reduce labeled data requirements

Consistency in demonstrations enhances model performance

Abstract

Recent advances in prompt-based learning have shown strong results on few-shot text classification by using cloze-style templates. Similar attempts have been made on named entity recognition (NER) which manually design templates to predict entity types for every text span in a sentence. However, such methods may suffer from error propagation induced by entity span detection, high cost due to enumeration of all possible text spans, and omission of inter-dependencies among token labels in a sentence. Here we present a simple demonstration-based learning method for NER, which lets the input be prefaced by task demonstrations for in-context learning. We perform a systematic study on demonstration strategy regarding what to include (entity examples, with or without surrounding context), how to select the examples, and what templates to use. Results on in-domain learning and domain adaptation show that the model's performance in low-resource settings can be largely improved with a suitable demonstration strategy (e.g., a 4-17% improvement on 25 train instances). We also find that good demonstration can save many labeled examples and consistency in demonstration contributes to better performance.