Scalable Deep Metric Learning on Attributed Graphs

TL;DR

This paper introduces scalable deep metric learning methods for attributed graphs, enabling efficient embeddings for large graphs and multiple tasks, with theoretical analysis and superior experimental results.

Contribution

It extends deep metric and contrastive learning to attributed graphs with mini-batch scalability and provides novel theoretical insights linking tuplet loss to contrastive learning.

Findings

High scalability in representation construction

Improved performance on node clustering, classification, and link prediction

Theoretical generalization bounds for downstream tasks

Abstract

We consider the problem of constructing embeddings of large attributed graphs and supporting multiple downstream learning tasks. We develop a graph embedding method, which is based on extending deep metric and unbiased contrastive learning techniques to 1) work with attributed graphs, 2) enabling a mini-batch based approach, and 3) achieving scalability. Based on a multi-class tuplet loss function, we present two algorithms -- DMT for semi-supervised learning and DMAT-i for the unsupervised case. Analyzing our methods, we provide a generalization bound for the downstream node classification task and for the first time relate tuplet loss to contrastive learning. Through extensive experiments, we show high scalability of representation construction, and in applying the method for three downstream tasks (node clustering, node classification, and link prediction) better consistency over any single existing method.