Implications of sparsity and high triangle density for graph representation learning

TL;DR

This paper investigates how sparsity and triangle density in graphs affect the ability to learn accurate node representations, revealing limitations of finite-dimensional models and proposing infinite-dimensional approaches.

Contribution

It demonstrates that sparse, triangle-rich graphs require infinite-dimensional models for accurate representation, challenging the idea that triangles always indicate community structure.

Findings

Finite-dimensional models cannot reproduce certain sparse, triangle-dense graphs.

Infinite-dimensional models with low-dimensional manifolds can represent these graphs.

Local neighborhoods can be represented with lower-dimensional embeddings.

Abstract

Recent work has shown that sparse graphs containing many triangles cannot be reproduced using a finite-dimensional representation of the nodes, in which link probabilities are inner products. Here, we show that such graphs can be reproduced using an infinite-dimensional inner product model, where the node representations lie on a low-dimensional manifold. Recovering a global representation of the manifold is impossible in a sparse regime. However, we can zoom in on local neighbourhoods, where a lower-dimensional representation is possible. As our constructions allow the points to be uniformly distributed on the manifold, we find evidence against the common perception that triangles imply community structure.