Let Me Check the Examples: Enhancing Demonstration Learning via Explicit Imitation

TL;DR

This paper introduces Imitation-Demo, a novel demonstration learning approach that explicitly imitates human review behavior using contrastive learning and label re-prediction, significantly improving performance across multiple classification tasks.

Contribution

The paper proposes a new imitation-based demonstration learning method that enhances prompt-demonstration dependency modeling and knowledge consolidation, outperforming existing methods.

Findings

Achieves state-of-the-art results on 11 out of 14 classification datasets.

Strengthens the association between prompts and demonstrations.

Demonstrates the effectiveness of imitation learning in demonstration-based NLP tasks.

Abstract

Demonstration learning aims to guide the prompt prediction via providing answered demonstrations in the few shot settings. Despite achieving promising results, existing work only concatenates the answered examples as demonstrations to the prompt template (including the raw context) without any additional operation, neglecting the prompt-demonstration dependencies. Besides, prior research found that randomly replacing the labels of demonstrations marginally hurts performance, illustrating that the model could not properly learn the knowledge brought by the demonstrations. Inspired by the human learning process, in this paper, we introduce Imitation DEMOnstration Learning (Imitation-Demo) to strengthen demonstration learning via explicitly imitating human review behaviour, which includes: (1) contrastive learning mechanism to concentrate on the similar demonstrations. (2) demonstration-label re-prediction method to consolidate known knowledge. Experiment results show that our proposed method achieves state-of-the-art performance on 11 out of 14 classification corpora. Further studies also prove that Imitation-Demo strengthen the association between prompt and demonstrations, which could provide the basis for exploring how demonstration learning works.