Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks

TL;DR

This paper introduces a minimally-supervised text categorization framework that leverages the structure-rich Web data by jointly training text understanding and network learning modules, achieving high accuracy with minimal labeled data.

Contribution

The novel framework combines network-based analysis with deep textual modeling in a co-training setup, enabling effective categorization with very few seed documents.

Findings

Achieves 92% accuracy with only three seed documents per category.

Outperforms existing methods significantly in minimal supervision settings.

Close to supervised BERT performance trained on 50K labels.

Abstract

Text categorization is an essential task in Web content analysis. Considering the ever-evolving Web data and new emerging categories, instead of the laborious supervised setting, in this paper, we focus on the minimally-supervised setting that aims to categorize documents effectively, with a couple of seed documents annotated per category. We recognize that texts collected from the Web are often structure-rich, i.e., accompanied by various metadata. One can easily organize the corpus into a text-rich network, joining raw text documents with document attributes, high-quality phrases, label surface names as nodes, and their associations as edges. Such a network provides a holistic view of the corpus' heterogeneous data sources and enables a joint optimization for network-based analysis and deep textual model training. We therefore propose a novel framework for minimally supervised categorization by learning from the text-rich network. Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning. Each module generates pseudo training labels from the unlabeled document set, and both modules mutually enhance each other by co-training using pooled pseudo labels. We test our model on two real-world datasets. On the challenging e-commerce product categorization dataset with 683 categories, our experiments show that given only three seed documents per category, our framework can achieve an accuracy of about 92%, significantly outperforming all compared methods; our accuracy is only less than 2% away from the supervised BERT model trained on about 50K labeled documents.