Injective flows for star-like manifolds
Marcello Massimo Negri, Jonathan Aellen, Volker Roth
TL;DR
This paper introduces a method for exact and efficient computation of Jacobian determinants in injective flows on star-like manifolds, enabling improved density estimation and variational inference without approximation overhead.
Contribution
We propose a novel class of injective flows tailored for star-like manifolds, allowing exact Jacobian computation at the same cost as normalizing flows, with applications in Bayesian and mixture models.
Findings
Exact Jacobian computation for star-like manifolds demonstrated
Enhanced density estimation in variational inference settings
Applications to Bayesian penalized likelihood and mixture models
Abstract
Normalizing Flows (NFs) are powerful and efficient models for density estimation. When modeling densities on manifolds, NFs can be generalized to injective flows but the Jacobian determinant becomes computationally prohibitive. Current approaches either consider bounds on the log-likelihood or rely on some approximations of the Jacobian determinant. In contrast, we propose injective flows for star-like manifolds and show that for such manifolds we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs. This aspect is particularly relevant for variational inference settings, where no samples are available and only some unnormalized target is known. Among many, we showcase the relevance of modeling densities on star-like manifolds in two settings. Firstly, we introduce a novel Objective Bayesian approach for penalized likelihood models by interpreting…
Peer Reviews
Decision·ICLR 2025 Poster
- The general idea of focusing on this class of manifolds, which has desired properties, is both simple and interesting, and the technical aspects appear solid. - Based on the experiments conducted, the proposed model supports the claims. - The paper is generally well-written and well-structured (see Questions for some suggestions).
- Even if this class of manifolds is useful for density modeling in the considered scenarios, it becomes somewhat restrictive if the $r(\theta)$ is not trainable. - Further experiments and/or comparisons with related works, such as in a maximum likelihood setting, could have been beneficial for demonstrating both the effectiveness and efficiency of the approach.
1. Paper is well written and readable. 2. The method is clearly explained and understandable, and it improves over sampling in the ambient dimension. 3. Claims are supported with proof and numerical evidence.
1. The method only applies to star-shaped manifolds embedded in one dimension higher. The authors acknowledge the limitation that their proof cannot be extended to manifolds with higher co-dimension. This is the main concern with the method - since this method only applies to a very specific class of co-dimension 1 submanifolds, I wonder about its applicability to problems of general interest. Since the method applies to a very specific class of manifolds instead of a more general class, the ben
- The proposed injective flow method for star-like manifolds significantly reduces the standard complexity of $O(d^3)$ to $O(d^2)$. - The injective flow approach appears to be both intriguing and novel, and the proof presented is sound. - The paper outlines several potential applications, including the objective Bayesian approach and posterior inference in probabilistic mixing models.
- Experiments are relatively limited, and no baselines are employed for comparison apart from the MCMC sampler in FIgure 9. I would suggest state-of-the-art normalizing flow methods and possibly other generative models for the baselines. - The purported efficiency enhancement from $O(d^3)$ to $O(d^2)$ is not substantiated by experimental results. The authors could include runtime comparisons between their method and standard normalizing flows for increasing dimensionality $d$. This would provide
Videos
Taxonomy
TopicsGeometric Analysis and Curvature Flows · Advanced Numerical Analysis Techniques · Algebraic Geometry and Number Theory
MethodsVariational Inference