Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning

TL;DR

This paper investigates the challenges of vector quantization in graph representation learning, identifies causes of codebook collapse, and proposes RGVQ, a regularization framework that improves codebook utilization and graph token diversity.

Contribution

It provides the first empirical analysis of codebook collapse in graph VQ, offers a theoretical understanding, and introduces RGVQ to enhance graph VQ performance.

Findings

RGVQ significantly improves codebook utilization.

RGVQ boosts performance across multiple graph tasks.

Codebook collapse occurs even with existing mitigation strategies.

Abstract

Vector Quantization (VQ) has recently emerged as a promising approach for learning discrete representations of graph-structured data. However, a fundamental challenge, i.e., codebook collapse, remains underexplored in the graph domain, significantly limiting the expressiveness and generalization of graph tokens.In this paper, we present the first empirical study showing that codebook collapse consistently occurs when applying VQ to graph data, even with mitigation strategies proposed in vision or language domains. To understand why graph VQ is particularly vulnerable to collapse, we provide a theoretical analysis and identify two key factors: early assignment imbalances caused by redundancy in graph features and structural patterns, and self-reinforcing optimization loops in deterministic VQ. To address these issues, we propose RGVQ, a novel framework that integrates graph topology and feature similarity as explicit regularization signals to enhance codebook utilization and promote token diversity. RGVQ introduces soft assignments via Gumbel-Softmax reparameterization, ensuring that all codewords receive gradient updates. In addition, RGVQ incorporates a structure-aware contrastive regularization to penalize the token co-assignments among dissimilar node pairs. Extensive experiments demonstrate that RGVQ substantially improves codebook utilization and consistently boosts the performance of state-of-the-art graph VQ backbones across multiple downstream tasks, enabling more expressive and transferable graph token representations.