Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification

TL;DR

This paper introduces a novel graph neural network model that learns sparse, dynamic document-level structures to improve inductive document classification, addressing limitations of static co-occurrence graphs.

Contribution

The paper proposes a trainable, sparse structure learning approach within GNNs for document classification, capturing dynamic contextual dependencies at the sentence level.

Findings

Outperforms state-of-the-art methods on multiple datasets.

Demonstrates the importance of learning sparse structures for each document.

Effectively captures local and global contextual information.

Abstract

Recently, graph neural networks (GNNs) have been widely used for document classification. However, most existing methods are based on static word co-occurrence graphs without sentence-level information, which poses three challenges:(1) word ambiguity, (2) word synonymity, and (3) dynamic contextual dependency. To address these challenges, we propose a novel GNN-based sparse structure learning model for inductive document classification. Specifically, a document-level graph is initially generated by a disjoint union of sentence-level word co-occurrence graphs. Our model collects a set of trainable edges connecting disjoint words between sentences and employs structure learning to sparsely select edges with dynamic contextual dependencies. Graphs with sparse structures can jointly exploit local and global contextual information in documents through GNNs. For inductive learning, the refined document graph is further fed into a general readout function for graph-level classification and optimization in an end-to-end manner. Extensive experiments on several real-world datasets demonstrate that the proposed model outperforms most state-of-the-art results, and reveal the necessity to learn sparse structures for each document.