Adversarial Erasing with Pruned Elements: Towards Better Graph Lottery Ticket

TL;DR

This paper introduces ACE-GLT, a novel framework that re-evaluates pruned graph components in Graph Lottery Tickets using adversarial erasing, leading to improved performance in graph neural network tasks.

Contribution

It proposes an adversarial complementary erasing framework to mine valuable information from pruned components, enhancing the quality of Graph Lottery Tickets beyond traditional magnitude-based pruning methods.

Findings

ACE-GLT outperforms existing GLT methods across various tasks.

Re-evaluating pruned components improves GNN efficiency.

The framework effectively mines valuable information from pruned edges and weights.

Abstract

Graph Lottery Ticket (GLT), a combination of core subgraph and sparse subnetwork, has been proposed to mitigate the computational cost of deep Graph Neural Networks (GNNs) on large input graphs while preserving original performance. However, the winning GLTs in exisiting studies are obtained by applying iterative magnitude-based pruning (IMP) without re-evaluating and re-considering the pruned information, which disregards the dynamic changes in the significance of edges/weights during graph/model structure pruning, and thus limits the appeal of the winning tickets. In this paper, we formulate a conjecture, i.e., existing overlooked valuable information in the pruned graph connections and model parameters which can be re-grouped into GLT to enhance the final performance. Specifically, we propose an adversarial complementary erasing (ACE) framework to explore the valuable information from the pruned components, thereby developing a more powerful GLT, referred to as the ACE-GLT. The main idea is to mine valuable information from pruned edges/weights after each round of IMP, and employ the ACE technique to refine the GLT processing. Finally, experimental results demonstrate that our ACE-GLT outperforms existing methods for searching GLT in diverse tasks. Our code will be made publicly available.