GraphVICRegHSIC: Towards improved self-supervised representation learning for graphs with a hyrbid loss function

TL;DR

This paper introduces GraphVICRegHSIC, a hybrid loss function for self-supervised graph representation learning, which outperforms existing methods on several datasets by combining VICReg and HSIC advantages.

Contribution

The paper proposes a novel hybrid loss function, GraphVICRegHSIC, that improves self-supervised graph learning by integrating VICReg and HSIC, and evaluates its effectiveness across multiple datasets.

Findings

Hybrid loss outperforms others in 4 of 7 datasets

Impact of batch size and augmentation strategies analyzed

Hybrid approach shows promising results for graph representation learning

Abstract

Self-supervised learning and pre-training strategieshave developed over the last few years especiallyfor Convolutional Neural Networks (CNNs). Re-cently application of such methods can also be no-ticed for Graph Neural Networks (GNNs) . In thispaper, we have used a graph based self-supervisedlearning strategy with different loss functions (Bar-low Twins[Zbontaret al., 2021], HSIC[Tsaiet al.,2021], VICReg[Bardeset al., 2021]) which haveshown promising results when applied with CNNspreviously. We have also proposed a hybrid lossfunction combining the advantages of VICReg andHSIC and called it as VICRegHSIC. The perfor-mance of these aforementioned methods have beencompared when applied to 7 different datasets suchas MUTAG, PROTEINS, IMDB-Binary, etc. Ex-periments showed that our hybrid loss function per-formed better than the remaining ones in 4 out of7 cases. Moreover, the impact of different batchsizes, projector dimensions and data augmentationstrategies have also been explored.