PINF: Continuous Normalizing Flows for Physics-Constrained Deep Learning
Feng Liu, Faguo Wu, Xiao Zhang
TL;DR
PINF introduces a physics-informed extension of continuous normalizing flows that efficiently solves high-dimensional Fokker-Planck equations by incorporating diffusion and leveraging the change of variables formula.
Contribution
The paper presents a novel mesh-free, causality-free method that combines normalizing flows with physics constraints to solve complex Fokker-Planck equations.
Findings
Efficiently solves high-dimensional Fokker-Planck equations.
Incorporates physics constraints into normalizing flows.
Mesh-free and causality-free approach.
Abstract
The normalization constraint on probability density poses a significant challenge for solving the Fokker-Planck equation. Normalizing Flow, an invertible generative model leverages the change of variables formula to ensure probability density conservation and enable the learning of complex data distributions. In this paper, we introduce Physics-Informed Normalizing Flows (PINF), a novel extension of continuous normalizing flows, incorporating diffusion through the method of characteristics. Our method, which is mesh-free and causality-free, can efficiently solve high dimensional time-dependent and steady-state Fokker-Planck equations.
Peer Reviews
Decision·ICLR 2024 Conference Withdrawn Submission
The paper is well grounded theoretically in a natural extension to CNFs which are applied specifically to the TFP and SFP problems.
Though the results are interesting, there seems to be a lack of any sort of baseline compared to say standard PINNs (which the authors mention as falling after d=3). Most of the results are evaluated qualitatively where the authors observe good agreements between the true solution and the nn solution. A way to perhaps solidify results could be to analyze some sort of loss metric (perhaps average MAPE or max MAPE) for different models (PINF, PINN) as d is increased, and see the differences in dro
1. PINF addresses the normalization constraint effectively, a challenge often faced in high-dimensional FP problems 2. The model shows effectiveness over a few questions including high-dimensional cases.
1. The problem formulation is not clear. There are 3 problem setups w./w.o. time varying/diffusion term. The drifting term seems to be a known function rather than to be learned. So what’s the point of the 1st setup (algorithm 1) here solving an PDE with explicit knowledge of time derivative equation? 2. The paper motivation is not clear. As the data is generated by solving the ODEs, what are the benefits to use neural network to replace PDE solvers? Whether NN is faster/more robust/generalizab
- Framing the problem as regression is clever – reminiscent of score matching, the technique behind diffusion models. Thus the central contribution of the paper is novel and interesting - Most of the exposition of the background and theory is good, see weaknesses below for examples where this is not true - The experiments shown seem to have worked very well. However, they are too easy (see below)
- Quite hard to understand some key points on a first reading. I only understood what was going on after looking at the algorithms. For example, it would have been very helpful to mention explicitly that by $\log p_{net}$ you mean $\phi$ (or just write $\phi$ in the equation instead of $\log p_{net}$) - Doesn’t mention that the divergence of a network (trace of gradient) is only cheap (constant wrt data dimension) if you are happy with an approximation via the Hutchinson trace estimator. Otherwi
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Gaussian Processes and Bayesian Inference · Generative Adversarial Networks and Image Synthesis
MethodsNormalizing Flows · Diffusion