Balancing Symmetry and Efficiency in Graph Flow Matching

TL;DR

This paper explores the trade-off between symmetry and efficiency in graph generative models, proposing a controllable symmetry modulation scheme that improves training speed and performance.

Contribution

It introduces a novel symmetry modulation method that relaxes equivariance during training, balancing convergence speed and overfitting in graph flow matching models.

Findings

Symmetry-breaking accelerates early training.

Proper modulation delays overfitting.

Achieves 19% of baseline training epochs for strong performance.

Abstract

Equivariance is central to graph generative models, as it ensures the model respects the permutation symmetry of graphs. However, strict equivariance can increase computational cost due to added architectural constraints, and can slow down convergence because the model must be consistent across a large space of possible node permutations. We study this trade-off for graph generative models. Specifically, we start from an equivariant discrete flow-matching model, and relax its equivariance during training via a controllable symmetry modulation scheme based on sinusoidal positional encodings and node permutations. Experiments first show that symmetry-breaking can accelerate early training by providing an easier learning signal, but at the expense of encouraging shortcut solutions that can cause overfitting, where the model repeatedly generates graphs that are duplicates of the training set. On the contrary, properly modulating the symmetry signal can delay overfitting while accelerating convergence, allowing the model to reach stronger performance with $19\%$ of the baseline training epochs.