Boosting Short Text Classification with Multi-Source Information Exploration and Dual-Level Contrastive Learning

TL;DR

This paper introduces MI-DELIGHT, a novel model that enhances short text classification by leveraging multi-source information, graph-based representations, and dual-level contrastive learning within a hierarchical architecture, outperforming existing models and large language models.

Contribution

The paper presents a new model combining multi-source information exploration, graph learning, and hierarchical dual-level contrastive learning for improved short text classification.

Findings

MI-DELIGHT outperforms previous models on benchmark datasets.

It surpasses popular large language models in several cases.

The hierarchical architecture effectively models task relationships.

Abstract

Short text classification, as a research subtopic in natural language processing, is more challenging due to its semantic sparsity and insufficient labeled samples in practical scenarios. We propose a novel model named MI-DELIGHT for short text classification in this work. Specifically, it first performs multi-source information (i.e., statistical information, linguistic information, and factual information) exploration to alleviate the sparsity issues. Then, the graph learning approach is adopted to learn the representation of short texts, which are presented in graph forms. Moreover, we introduce a dual-level (i.e., instance-level and cluster-level) contrastive learning auxiliary task to effectively capture different-grained contrastive information within massive unlabeled data. Meanwhile, previous models merely perform the main task and auxiliary tasks in parallel, without considering the relationship among tasks. Therefore, we introduce a hierarchical architecture to explicitly model the correlations between tasks. We conduct extensive experiments across various benchmark datasets, demonstrating that MI-DELIGHT significantly surpasses previous competitive models. It even outperforms popular large language models on several datasets.