Fast Graph Learning with Unique Optimal Solutions

TL;DR

This paper introduces a fast, closed-form solution for graph representation learning tasks using a novel SVD implementation, achieving competitive results with significantly reduced computation time.

Contribution

The authors propose a linearized, Frobenius norm-based approach for GRL models that allows for closed-form solutions and efficient SVD computation, enabling faster training.

Findings

Achieves competitive performance on node classification and link prediction tasks.

Provides orders of magnitude speedup over traditional methods.

Open-sourced implementation in TensorFlow for practical use.

Abstract

We consider two popular Graph Representation Learning (GRL) methods: message passing for node classification and network embedding for link prediction. For each, we pick a popular model that we: (i) linearize and (ii) and switch its training objective to Frobenius norm error minimization. These simplifications can cast the training into finding the optimal parameters in closed-form. We program in TensorFlow a functional form of Truncated Singular Value Decomposition (SVD), such that, we could decompose a dense matrix $\mathbf{M}$, without explicitly computing $\mathbf{M}$. We achieve competitive performance on popular GRL tasks while providing orders of magnitude speedup. We open-source our code at http://github.com/samihaija/tf-fsvd