On the Effectiveness of Hybrid Pooling in Mixup-Based Graph Learning for Language Processing

TL;DR

This paper investigates how different graph pooling operators influence the effectiveness of Manifold-Mixup data augmentation in GNNs for language processing, revealing that hybrid pooling operators improve model accuracy and robustness.

Contribution

It provides the first comprehensive empirical analysis of the impact of various graph pooling operators on Mixup-based graph learning performance.

Findings

Hybrid pooling operators outperform Max-pooling and GMT in accuracy.

Hybrid pooling enhances robustness of GNN models.

Empirical results on language datasets confirm effectiveness.

Abstract

Graph neural network (GNN)-based graph learning has been popular in natural language and programming language processing, particularly in text and source code classification. Typically, GNNs are constructed by incorporating alternating layers which learn transformations of graph node features, along with graph pooling layers that use graph pooling operators (e.g., Max-pooling) to effectively reduce the number of nodes while preserving the semantic information of the graph. Recently, to enhance GNNs in graph learning tasks, Manifold-Mixup, a data augmentation technique that produces synthetic graph data by linearly mixing a pair of graph data and their labels, has been widely adopted. However, the performance of Manifold-Mixup can be highly affected by graph pooling operators, and there have not been many studies that are dedicated to uncovering such affection. To bridge this gap, we take an early step to explore how graph pooling operators affect the performance of Mixup-based graph learning. To that end, we conduct a comprehensive empirical study by applying Manifold-Mixup to a formal characterization of graph pooling based on 11 graph pooling operations (9 hybrid pooling operators, 2 non-hybrid pooling operators). The experimental results on both natural language datasets (Gossipcop, Politifact) and programming language datasets (JAVA250, Python800) demonstrate that hybrid pooling operators are more effective for Manifold-Mixup than the standard Max-pooling and the state-of-the-art graph multiset transformer (GMT) pooling, in terms of producing more accurate and robust GNN models.