Human in the loop: How to effectively create coherent topics by manually labeling only a few documents per class

TL;DR

This paper demonstrates that supervised few-shot learning combined with simple topic extraction can produce more coherent topics than traditional unsupervised methods, even with minimal labeled data.

Contribution

It introduces a novel approach integrating supervised few-shot learning with simple topic extraction to improve topic coherence with limited labels.

Findings

Supervised few-shot learning outperforms unsupervised topic models in coherence.

Few labeled documents per class are sufficient for effective topic creation.

The method simplifies the process while maintaining high-quality topic coherence.

Abstract

Few-shot methods for accurate modeling under sparse label-settings have improved significantly. However, the applications of few-shot modeling in natural language processing remain solely in the field of document classification. With recent performance improvements, supervised few-shot methods, combined with a simple topic extraction method pose a significant challenge to unsupervised topic modeling methods. Our research shows that supervised few-shot learning, combined with a simple topic extraction method, can outperform unsupervised topic modeling techniques in terms of generating coherent topics, even when only a few labeled documents per class are used.