GNN-CNN: An Efficient Hybrid Model of Convolutional and Graph Neural Networks for Text Representation

TL;DR

This paper presents a hybrid GNN-CNN model that efficiently processes long texts by combining graph structures and convolutional features, outperforming transformers in speed and maintaining competitive accuracy.

Contribution

The study introduces a novel GNN-CNN architecture with real-time graph generation and LLM integration, enhancing efficiency and performance for long text classification tasks.

Findings

Achieves high efficiency with real-time graph generation.

Maintains competitive accuracy compared to state-of-the-art models.

Demonstrates structural semantic properties in generated graphs.

Abstract

Time, cost, and energy efficiency are critical considerations in Deep-Learning (DL), particularly when processing long texts. Transformers, which represent the current state of the art, exhibit quadratic computational complexity relative to input length, making them inefficient for extended documents. This study introduces a novel model architecture that combines Graph Neural Networks (GNNs) and Convolutional Neural Networks (CNNs), integrated with a real-time, end-to-end graph generation mechanism. The model processes compact batches of character-level inputs without requiring padding or truncation. To enhance performance while maintaining high speed and efficiency, the model incorporates information from Large Language Models (LLMs), such as token embeddings and sentiment polarities, through efficient dictionary lookups. It captures local contextual patterns using CNNs, expands local receptive fields via lattice-based graph structures, and employs small-world graphs to aggregate document-level information. The generated graphs exhibit structural properties indicative of meaningful semantic organization, with an average clustering coefficient of approximately 0.45 and an average shortest path length ranging between 4 and 5. The model is evaluated across multiple text classification tasks, including sentiment analysis and news-categorization, and is compared against state-of-the-art models. Experimental results confirm the proposed model's efficiency and competitive performance.