Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions

TL;DR

Gen-Z introduces a generative zero-shot text classification framework that leverages contextualized label descriptions, improving robustness and personalization over traditional prompting methods across multiple benchmarks.

Contribution

It proposes a novel generative prompting approach that incorporates contextual label descriptions, enhancing zero-shot classification performance and robustness.

Findings

Outperforms zero-shot and few-shot baselines on standard benchmarks.

Improves robustness to prompt variations.

Enables personalized classification through contextual label descriptions.

Abstract

Language model (LM) prompting--a popular paradigm for solving NLP tasks--has been shown to be susceptible to miscalibration and brittleness to slight prompt variations, caused by its discriminative prompting approach, i.e., predicting the label given the input. To address these issues, we propose Gen-Z--a generative prompting framework for zero-shot text classification. GEN-Z is generative, as it measures the LM likelihood of input text, conditioned on natural language descriptions of labels. The framework is multivariate, as label descriptions allow us to seamlessly integrate additional contextual information about the labels to improve task performance. On various standard classification benchmarks, with six open-source LM families, we show that zero-shot classification with simple contextualization of the data source of the evaluation set consistently outperforms both zero-shot and few-shot baselines while improving robustness to prompt variations. Further, our approach enables personalizing classification in a zero-shot manner by incorporating author, subject, or reader information in the label descriptions.