InducT-GCN: Inductive Graph Convolutional Networks for Text Classification

TL;DR

InducT-GCN introduces an inductive graph convolutional network for text classification that performs well without relying on pre-trained resources or transductive assumptions, demonstrating scalability and superior results across multiple benchmarks.

Contribution

The paper proposes a novel inductive GCN framework for text classification that operates solely on training data statistics, eliminating the need for pre-trained embeddings or test data during training.

Findings

Outperforms state-of-the-art transductive and resource-dependent models on five benchmarks.

Reduces time and space complexity with increased data size.

Effective in scenarios with limited training data and no external resources.

Abstract

Text classification aims to assign labels to textual units by making use of global information. Recent studies have applied graph neural network (GNN) to capture the global word co-occurrence in a corpus. Existing approaches require that all the nodes (training and test) in a graph are present during training, which are transductive and do not naturally generalise to unseen nodes. To make those models inductive, they use extra resources, like pretrained word embedding. However, high-quality resource is not always available and hard to train. Under the extreme settings with no extra resource and limited amount of training set, can we still learn an inductive graph-based text classification model? In this paper, we introduce a novel inductive graph-based text classification framework, InducT-GCN (InducTive Graph Convolutional Networks for Text classification). Compared to transductive models that require test documents in training, we construct a graph based on the statistics of training documents only and represent document vectors with a weighted sum of word vectors. We then conduct one-directional GCN propagation during testing. Across five text classification benchmarks, our InducT-GCN outperformed state-of-the-art methods that are either transductive in nature or pre-trained additional resources. We also conducted scalability testing by gradually increasing the data size and revealed that our InducT-GCN can reduce the time and space complexity. The code is available on: https://github.com/usydnlp/InductTGCN.