Continual Graph Convolutional Network for Text Classification

TL;DR

This paper introduces ContGCN, a continual graph convolutional network for text classification that dynamically updates its graph structure during training and testing, enabling online inference on streaming data.

Contribution

It proposes a novel all-token-any-document paradigm and a memory-augmented contrastive learning method for continual GCN training and inference.

Findings

8.86% performance improvement in online system

Enhanced inference quality on public datasets

Effective in streaming text data scenarios

Abstract

Graph convolutional network (GCN) has been successfully applied to capture global non-consecutive and long-distance semantic information for text classification. However, while GCN-based methods have shown promising results in offline evaluations, they commonly follow a seen-token-seen-document paradigm by constructing a fixed document-token graph and cannot make inferences on new documents. It is a challenge to deploy them in online systems to infer steaming text data. In this work, we present a continual GCN model (ContGCN) to generalize inferences from observed documents to unobserved documents. Concretely, we propose a new all-token-any-document paradigm to dynamically update the document-token graph in every batch during both the training and testing phases of an online system. Moreover, we design an occurrence memory module and a self-supervised contrastive learning objective to update ContGCN in a label-free manner. A 3-month A/B test on Huawei public opinion analysis system shows ContGCN achieves 8.86% performance gain compared with state-of-the-art methods. Offline experiments on five public datasets also show ContGCN can improve inference quality. The source code will be released at https://github.com/Jyonn/ContGCN.