Retrieval Augmented Zero-Shot Text Classification

TL;DR

QZero is a training-free method that enhances zero-shot text classification by retrieving supporting Wikipedia categories, significantly improving performance without retraining, especially in resource-constrained settings.

Contribution

Introducing QZero, a novel knowledge augmentation technique that reformulates queries using Wikipedia retrieval to boost zero-shot classification performance without additional training.

Findings

QZero improves classification accuracy by at least 5% in news and medical datasets.

It enables small embedding models to match larger models' performance.

QZero provides insights into query context and topic relevance.

Abstract

Zero-shot text learning enables text classifiers to handle unseen classes efficiently, alleviating the need for task-specific training data. A simple approach often relies on comparing embeddings of query (text) to those of potential classes. However, the embeddings of a simple query sometimes lack rich contextual information, which hinders the classification performance. Traditionally, this has been addressed by improving the embedding model with expensive training. We introduce QZero, a novel training-free knowledge augmentation approach that reformulates queries by retrieving supporting categories from Wikipedia to improve zero-shot text classification performance. Our experiments across six diverse datasets demonstrate that QZero enhances performance for state-of-the-art static and contextual embedding models without the need for retraining. Notably, in News and medical topic classification tasks, QZero improves the performance of even the largest OpenAI embedding model by at least 5% and 3%, respectively. Acting as a knowledge amplifier, QZero enables small word embedding models to achieve performance levels comparable to those of larger contextual models, offering the potential for significant computational savings. Additionally, QZero offers meaningful insights that illuminate query context and verify topic relevance, aiding in understanding model predictions. Overall, QZero improves embedding-based zero-shot classifiers while maintaining their simplicity. This makes it particularly valuable for resource-constrained environments and domains with constantly evolving information.