Entailment as Few-Shot Learner

TL;DR

This paper introduces EFL, a method that transforms NLP tasks into entailment problems and fine-tunes small language models with minimal data, significantly enhancing their few-shot learning capabilities.

Contribution

The paper presents EFL, a novel approach that reformulates NLP tasks as entailment problems and improves small LMs' few-shot learning performance with minimal data and easy extensions.

Findings

EFL improves few-shot learning performance by 12% over existing methods.

EFL achieves competitive results with much larger models like GPT-3.

The approach can be combined with contrastive data augmentation and extended to multilingual tasks.

Abstract

Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners. However, their success hinges largely on scaling model parameters to a degree that makes it challenging to train and serve. In this paper, we propose a new approach, named as EFL, that can turn small LMs into better few-shot learners. The key idea of this approach is to reformulate potential NLP task into an entailment one, and then fine-tune the model with as little as 8 examples. We further demonstrate our proposed method can be: (i) naturally combined with an unsupervised contrastive learning-based data augmentation method; (ii) easily extended to multilingual few-shot learning. A systematic evaluation on 18 standard NLP tasks demonstrates that this approach improves the various existing SOTA few-shot learning methods by 12\%, and yields competitive few-shot performance with 500 times larger models, such as GPT-3.