Weakly Supervised Text Classification using Supervision Signals from a Language Model

TL;DR

This paper introduces a weakly supervised text classification method that leverages masked language models with prompt-based supervision signals, outperforming baselines on multiple datasets without requiring annotated data.

Contribution

It proposes a novel approach using language model-generated supervision signals via prompts, enabling effective classification without labeled data.

Findings

Outperforms baselines by 2-4% on three datasets

Uses prompt-based supervision signals from masked language models

Does not require annotated training data

Abstract

Solving text classification in a weakly supervised manner is important for real-world applications where human annotations are scarce. In this paper, we propose to query a masked language model with cloze style prompts to obtain supervision signals. We design a prompt which combines the document itself and "this article is talking about [MASK]." A masked language model can generate words for the [MASK] token. The generated words which summarize the content of a document can be utilized as supervision signals. We propose a latent variable model to learn a word distribution learner which associates generated words to pre-defined categories and a document classifier simultaneously without using any annotated data. Evaluation on three datasets, AGNews, 20Newsgroups, and UCINews, shows that our method can outperform baselines by 2%, 4%, and 3%.