Hodge-Aware Contrastive Learning

TL;DR

This paper introduces a spectral-aware contrastive learning method for simplicial data, leveraging Hodge decomposition to improve embeddings for higher-order network structures, outperforming supervised methods.

Contribution

It develops a novel contrastive learning framework that incorporates spectral properties via Hodge decomposition for simplicial complex data.

Findings

Superior performance on edge flow classification tasks

Effective encoding of spectral invariances in embeddings

Enhanced separation of dissimilar instances in spectral space

Abstract

Simplicial complexes prove effective in modeling data with multiway dependencies, such as data defined along the edges of networks or within other higher-order structures. Their spectrum can be decomposed into three interpretable subspaces via the Hodge decomposition, resulting foundational in numerous applications. We leverage this decomposition to develop a contrastive self-supervised learning approach for processing simplicial data and generating embeddings that encapsulate specific spectral information.Specifically, we encode the pertinent data invariances through simplicial neural networks and devise augmentations that yield positive contrastive examples with suitable spectral properties for downstream tasks. Additionally, we reweight the significance of negative examples in the contrastive loss, considering the similarity of their Hodge components to the anchor. By encouraging a stronger separation among less similar instances, we obtain an embedding space that reflects the spectral properties of the data. The numerical results on two standard edge flow classification tasks show a superior performance even when compared to supervised learning techniques. Our findings underscore the importance of adopting a spectral perspective for contrastive learning with higher-order data.