To Share or Not to Share: Investigating Weight Sharing in Variational Graph Autoencoders

TL;DR

This paper systematically examines weight sharing in variational graph autoencoders, revealing that its benefits in optimization and regularization generally surpass drawbacks, thus supporting its adoption.

Contribution

It provides a comprehensive analysis and extensive experimental validation of weight sharing in VGAEs, establishing its effectiveness and practical advantages.

Findings

Weight sharing improves VGAE training stability

Weight sharing enhances model regularization

Weight sharing does not significantly reduce performance

Abstract

This paper investigates the understudied practice of weight sharing (WS) in variational graph autoencoders (VGAE). WS presents both benefits and drawbacks for VGAE model design and node embedding learning, leaving its overall relevance unclear and the question of whether it should be adopted unresolved. We rigorously analyze its implications and, through extensive experiments on a wide range of graphs and VGAE variants, demonstrate that the benefits of WS consistently outweigh its drawbacks. Based on our findings, we recommend WS as an effective approach to optimize, regularize, and simplify VGAE models without significant performance loss.