Scalable Deep Graph Clustering with Random-walk based Self-supervised Learning

TL;DR

This paper introduces RwSL, a scalable deep graph clustering method that leverages random-walk based filtering and self-supervised learning, capable of handling web-scale graphs with over a million nodes and billions of edges.

Contribution

The paper proposes a novel scalable deep clustering algorithm, RwSL, that relates Laplacian smoothing to Generalized PageRank and uses random-walk filtering for large-scale graphs.

Findings

RwSL outperforms recent baselines on 6 real-world datasets.

It can scale beyond 1 million nodes, handling web-scale graphs.

Demonstrated clustering on a graph with 1.8 billion edges using a single GPU.

Abstract

Web-based interactions can be frequently represented by an attributed graph, and node clustering in such graphs has received much attention lately. Multiple efforts have successfully applied Graph Convolutional Networks (GCN), though with some limits on accuracy as GCNs have been shown to suffer from over-smoothing issues. Though other methods (particularly those based on Laplacian Smoothing) have reported better accuracy, a fundamental limitation of all the work is a lack of scalability. This paper addresses this open problem by relating the Laplacian smoothing to the Generalized PageRank and applying a random-walk based algorithm as a scalable graph filter. This forms the basis for our scalable deep clustering algorithm, RwSL, where through a self-supervised mini-batch training mechanism, we simultaneously optimize a deep neural network for sample-cluster assignment distribution and an autoencoder for a clustering-oriented embedding. Using 6 real-world datasets and 6 clustering metrics, we show that RwSL achieved improved results over several recent baselines. Most notably, we show that RwSL, unlike all other deep clustering frameworks, can continue to scale beyond graphs with more than one million nodes, i.e., handle web-scale. We also demonstrate how RwSL could perform node clustering on a graph with 1.8 billion edges using only a single GPU.