The Impact of Dimensionality on the Stability of Node Embeddings

TL;DR

This paper investigates how the dimensionality of node embeddings affects their stability and performance across various methods and datasets, revealing complex trade-offs and method-dependent patterns.

Contribution

It systematically analyzes the impact of embedding dimension on stability and performance for five popular graph embedding methods, providing new insights into hyperparameter effects.

Findings

Embedding stability varies significantly with dimensionality.

Some methods become more stable with higher dimensions, others do not.

Maximum stability does not always coincide with best task performance.

Abstract

Previous work has established that neural network-based node embeddings return different outcomes when trained with identical parameters on the same dataset, just from using different training seeds. Yet, it has not been thoroughly analyzed how key hyperparameters such as embedding dimension could impact this instability. In this work, we investigate how varying the dimensionality of node embeddings influences both their stability and downstream performance. We systematically evaluate five widely used methods -- ASNE, DGI, GraphSAGE, node2vec, and VERSE -- across multiple datasets and embedding dimensions. We assess stability from both a representational perspective and a functional perspective, alongside performance evaluation. Our results show that embedding stability varies significantly with dimensionality, but we observe different patterns across the methods we consider: while some approaches, such as node2vec and ASNE, tend to become more stable with higher dimensionality, other methods do not exhibit the same trend. Moreover, we find that maximum stability does not necessarily align with optimal task performance. These findings highlight the importance of carefully selecting embedding dimension, and provide new insights into the trade-offs between stability, performance, and computational effectiveness in graph representation learning.