BASiS: Batch Aligned Spectral Embedding Space

TL;DR

This paper introduces BASiS, a novel method for learning spectral embeddings of graphs that ensures consistent eigenspace alignment across batches, improving stability and performance over state-of-the-art techniques.

Contribution

It proposes a stable batch alignment mechanism for spectral embedding learning, addressing inconsistency issues in batch spectral graph methods.

Findings

Improved NMI, ACC, and classification accuracy.

More stable spectral embeddings across batches.

Outperforms state-of-the-art methods in experiments.

Abstract

Graph is a highly generic and diverse representation, suitable for almost any data processing problem. Spectral graph theory has been shown to provide powerful algorithms, backed by solid linear algebra theory. It thus can be extremely instrumental to design deep network building blocks with spectral graph characteristics. For instance, such a network allows the design of optimal graphs for certain tasks or obtaining a canonical orthogonal low-dimensional embedding of the data. Recent attempts to solve this problem were based on minimizing Rayleigh-quotient type losses. We propose a different approach of directly learning the eigensapce. A severe problem of the direct approach, applied in batch-learning, is the inconsistent mapping of features to eigenspace coordinates in different batches. We analyze the degrees of freedom of learning this task using batches and propose a stable alignment mechanism that can work both with batch changes and with graph-metric changes. We show that our learnt spectral embedding is better in terms of NMI, ACC, Grassman distance, orthogonality and classification accuracy, compared to SOTA. In addition, the learning is more stable.