Soft Contextualized Encoder For User Defined Text Classification

TL;DR

This paper introduces a soft-contextualized encoder for user-defined text classification that effectively generalizes to unseen classes, achieving state-of-the-art results across multiple benchmarks.

Contribution

The paper presents a novel encoder architecture that incorporates label set context and soft prompts, enabling zero-shot classification for unseen classes in UDTC.

Findings

Achieves state-of-the-art performance on multiple UDTC benchmarks.

Effectively generalizes to zero-shot classification over unseen classes.

Outperforms or matches existing baselines across datasets.

Abstract

User-Defined Text Classification (UDTC) considers the challenge of classifying input text to user-specified, previously unseen classes, a setting that arises frequently in real-world applications such as enterprise analytics, content moderation, and domain-specific information retrieval. We propose a soft-contextualized encoder architecture for UDTC which contextualizes each candidate label with the label set and a static soft prompt representation of the input query. Training on diverse, multi-source datasets enables the model to generalize effectively to zero-shot classification over entirely unseen topic sets drawn from arbitrary domains. We evaluate the proposed architecture both on held-out in-distribution test data and on multiple unseen UDTC benchmarks. Across datasets, the model achieves state-of-the-art performance, consistently outperforming or matching the baselines.