The Benefits of Label-Description Training for Zero-Shot Text Classification

TL;DR

This paper introduces a simple label-description finetuning method that significantly improves zero-shot text classification accuracy and robustness across various datasets by using label descriptions instead of input texts.

Contribution

The authors propose a novel label-description finetuning approach that enhances zero-shot classification performance and robustness with minimal effort and data.

Findings

Improves zero-shot accuracy by 17-19% absolute across datasets.

Increases robustness to prompt and label mapping choices.

Yields models that perform well across multiple text domains.

Abstract

Pretrained language models have improved zero-shot text classification by allowing the transfer of semantic knowledge from the training data in order to classify among specific label sets in downstream tasks. We propose a simple way to further improve zero-shot accuracies with minimal effort. We curate small finetuning datasets intended to describe the labels for a task. Unlike typical finetuning data, which has texts annotated with labels, our data simply describes the labels in language, e.g., using a few related terms, dictionary/encyclopedia entries, and short templates. Across a range of topic and sentiment datasets, our method is more accurate than zero-shot by 17-19% absolute. It is also more robust to choices required for zero-shot classification, such as patterns for prompting the model to classify and mappings from labels to tokens in the model's vocabulary. Furthermore, since our data merely describes the labels but does not use input texts, finetuning on it yields a model that performs strongly on multiple text domains for a given label set, even improving over few-shot out-of-domain classification in multiple settings.