Small sample-based adaptive text classification through iterative and contrastive description refinement

TL;DR

This paper introduces a novel adaptive text classification framework that leverages iterative refinement, contrastive prompting, and active learning to improve zero-shot classification performance in dynamic, real-world environments.

Contribution

It presents a new framework combining iterative topic refinement, contrastive prompting, and human-in-the-loop updates for effective zero-shot text classification with minimal supervision.

Findings

Achieved 91% accuracy on AGNews with limited labeled data.

Maintained high accuracy with unseen categories, demonstrating robustness.

Effective in dynamic environments with evolving categories.

Abstract

Zero-shot text classification remains a difficult task in domains with evolving knowledge and ambiguous category boundaries, such as ticketing systems. Large language models (LLMs) often struggle to generalize in these scenarios due to limited topic separability, while few-shot methods are constrained by insufficient data diversity. We propose a classification framework that combines iterative topic refinement, contrastive prompting, and active learning. Starting with a small set of labeled samples, the model generates initial topic labels. Misclassified or ambiguous samples are then used in an iterative contrastive prompting process to refine category distinctions by explicitly teaching the model to differentiate between closely related classes. The framework features a human-in-the-loop component, allowing users to introduce or revise category definitions in natural language. This enables seamless integration of new, unseen categories without retraining, making the system well-suited for real-world, dynamic environments. The evaluations on AGNews and DBpedia demonstrate strong performance: 91% accuracy on AGNews (3 seen, 1 unseen class) and 84% on DBpedia (8 seen, 1 unseen), with minimal accuracy shift after introducing unseen classes (82% and 87%, respectively). The results highlight the effectiveness of prompt-based semantic reasoning for fine-grained classification with limited supervision.