Towards Scalable Topological Regularizers
Hiu-Tung Wong, Darrick Lee, Hong Yan
TL;DR
This paper introduces a scalable topological regularizer based on principal persistence measures and GPU acceleration, improving training stability and efficiency in tasks like shape matching and image generation.
Contribution
It proposes a novel, scalable topological regularizer using principal persistence measures, with a GPU implementation and proven gradient continuity for smooth densities.
Findings
Effective in shape matching tasks
Enhances image generation quality
Applicable to semi-supervised learning
Abstract
Latent space matching, which consists of matching distributions of features in latent space, is a crucial component for tasks such as adversarial attacks and defenses, domain adaptation, and generative modelling. Metrics for probability measures, such as Wasserstein and maximum mean discrepancy, are commonly used to quantify the differences between such distributions. However, these are often costly to compute, or do not appropriately take the geometric and topological features of the distributions into consideration. Persistent homology is a tool from topological data analysis which quantifies the multi-scale topological structure of point clouds, and has recently been used as a topological regularizer in learning tasks. However, computation costs preclude larger scale computations, and discontinuities in the gradient lead to unstable training behavior such as in adversarial tasks. We…
Peer Reviews
Decision·ICLR 2025 Poster
The paper pairs the proposed regularizer with a solid theoretical analysis of the smoothness of the resulting loss function, which justifies its use as an additional term to the (W)GAN loss. Experimental results show that the proposed regularizer provides a significant advantage in training GANs, especially for self-supervised tasks. Although the paper is theoretically dense, it can still be followed by non-experts in the PH fields. Nevertheless, it would be helpful to remark the purpose of e
The main weakness of the paper is the introduction and motivation of the problem and the relation of the proposed method to the current literature. From the introduction (and also the title), it isn’t really clear what problem the paper is going to tackle. While the introduction talks about general regularizers for latent space representations, most of the methodological development and the experimental section tackle specifically the problem of regularizing GANs. I would be clearer about this f
- Good introduction / preliminaries (section 1 and 2). - The work mixes results from different ideas (mix of MMD, persistence measures, etc.), and may be useful beyond the scope covered by this paper (improving GAN training). It showcases the use of PPM which are interesting objects in their own, introduce novel possibly useful metric between these topological descriptors, etc. - interesting experiments, mixing pedagogical Proof-of-Concept and more advanced experiments. - Nice animation in th
1. While well written, the introduction and section 2 somewhat fail to motivate the need to account for the geometry/topology when comparing distributions in the latent space. This is just given as a fact (line 142-143), but basically, why should one care about such information? Is there some situations in which this clearly lacks in GAN models? I understand that this needs is empirically confirmed in the experiment sections, but is there a good _a priori_ reason? 2. Somewhat in the same vein:
In general it is great to see a push towards scalable topological methods and this entails the core strengths of the paper, as it an impactful and relevant problem to work on. Often in TDA, the focal point lies in theoretical justification of the methods with only limited emperical evidence, mostly on small datasets. Lack of scalability of PH being the main driver of this phenomenon and therefore this paper is of importance in building the bridge in applying TDA in large scale applications. The
Where the theoretical contributions have good exposition, the implementation details are less thoroughly addressed. As one of the two cornerstones of the contributions, it would be nice to see where the authors differ in their implementation from previous work that allowed them to compute the PH on the GPU. The details seem to be lacking in the paper and together with the release of the code, would strengthen the paper considerably. For the experimental section the optimization of point clouds
Videos
Taxonomy
TopicsDigital Filter Design and Implementation · Neural Networks and Applications