Improving Graph-Based Text Representations with Character and Word Level N-grams

TL;DR

This paper introduces a novel heterogeneous text graph combining word and character n-grams with document nodes, along with two new graph neural models, leading to improved performance in text classification and summarization tasks.

Contribution

The paper proposes a new heterogeneous word-character text graph and two neural models, WCTextGCN and WCTextGAT, enhancing graph-based text representations.

Findings

Models outperform existing graph-based methods

Significant improvements in text classification accuracy

Enhanced results in automatic text summarization

Abstract

Graph-based text representation focuses on how text documents are represented as graphs for exploiting dependency information between tokens and documents within a corpus. Despite the increasing interest in graph representation learning, there is limited research in exploring new ways for graph-based text representation, which is important in downstream natural language processing tasks. In this paper, we first propose a new heterogeneous word-character text graph that combines word and character n-gram nodes together with document nodes, allowing us to better learn dependencies among these entities. Additionally, we propose two new graph-based neural models, WCTextGCN and WCTextGAT, for modeling our proposed text graph. Extensive experiments in text classification and automatic text summarization benchmarks demonstrate that our proposed models consistently outperform competitive baselines and state-of-the-art graph-based models.