FlexiDrop: Theoretical Insights and Practical Advances in Random Dropout Method on GNNs

TL;DR

FlexiDrop introduces a theoretically grounded, adaptive dropout method for GNNs that improves generalization and outperforms traditional approaches on benchmark datasets.

Contribution

The paper presents a novel dropout method for GNNs that unifies dropout rate and loss optimization, balancing model complexity and generalization.

Findings

FlexiDrop outperforms traditional dropout methods in GNNs.

Theoretical analysis links dropout rate to generalization error.

Adaptive dropout rate improves model robustness.

Abstract

Graph Neural Networks (GNNs) are powerful tools for handling graph-type data. Recently, GNNs have been widely applied in various domains, but they also face some issues, such as overfitting, over-smoothing and non-robustness. The existing research indicates that random dropout methods are an effective way to address these issues. However, random dropout methods in GNNs still face unresolved problems. Currently, the choice of dropout rate, often determined by heuristic or grid search methods, can increase the generalization error, contradicting the principal aims of dropout. In this paper, we propose a novel random dropout method for GNNs called FlexiDrop. First, we conduct a theoretical analysis of dropout in GNNs using rademacher complexity and demonstrate that the generalization error of traditional random dropout methods is constrained by a function related to the dropout rate. Subsequently, we use this function as a regularizer to unify the dropout rate and empirical loss within a single loss function, optimizing them simultaneously. Therefore, our method enables adaptive adjustment of the dropout rate and theoretically balances the trade-off between model complexity and generalization ability. Furthermore, extensive experimental results on benchmark datasets show that FlexiDrop outperforms traditional random dropout methods in GNNs.