Wavelet Diffusion Neural Operator
Peiyan Hu, Rui Wang, Xiang Zheng, Tao Zhang, Haodong Feng, Ruiqi Feng, Long Wei, Yue Wang, Zhi-Ming Ma, Tailin Wu
TL;DR
The Wavelet Diffusion Neural Operator (WDNO) advances PDE simulation and control by modeling in the wavelet domain and employing multi-resolution training, significantly improving accuracy and generalization across complex physical systems.
Contribution
WDNO introduces wavelet domain diffusion modeling and multi-resolution training, enhancing handling of abrupt changes and resolution generalization in PDE-based simulations.
Findings
Superior performance on multiple physical systems.
78% reduction in smoke leakage control error.
Effective long-term and detail prediction improvements.
Abstract
Simulating and controlling physical systems described by partial differential equations (PDEs) are crucial tasks across science and engineering. Recently, diffusion generative models have emerged as a competitive class of methods for these tasks due to their ability to capture long-term dependencies and model high-dimensional states. However, diffusion models typically struggle with handling system states with abrupt changes and generalizing to higher resolutions. In this work, we propose Wavelet Diffusion Neural Operator (WDNO), a novel PDE simulation and control framework that enhances the handling of these complexities. WDNO comprises two key innovations. Firstly, WDNO performs diffusion-based generative modeling in the wavelet domain for the entire trajectory to handle abrupt changes and long-term dependencies effectively. Secondly, to address the issue of poor generalization across…
Peer Reviews
Decision·ICLR 2025 Poster
1. A wavelet-based neural operator that surpasses the performance of exising methods. 2. Scaling of the input system reduces the solution manifold that the model needs to learn. 3. The paper is generally well-written, with details carefully reported.
The most significant weakness point of the work is the small number of temporal steps in all the test cases, which might actually be one of the limitation of these models, since the usual approach adopted to achieve long roll-out inference without instability is to introduce artificial training noise - it might be difficult to do so when one is already training a denoising model. It should be noted that the authors iteratively try to strengthen the point that they are doing "long-term dynamics"
* The main idea of combining WNO with diffusion is simple, novel and well justified. * Every design choice is justified with multiple tests and ablation studies. * The method outperforms other similar state-of-the-art techniques such as traditional U-Net/CNN to the newest Neural Operator architectures.
* The method is constrained to static uniform grid data and low-dimensional toy examples. * The generalization to more complicated tasks or boundary conditions might not be trivial.
1) Multi-resolution training 2) Diffusion in the wavelet domain to capture long-term dependencies and abrupt changes effectively.
1) The evaluation omits comparisons with state-of-the-art operators like Transolver, GNOT, LSM, DPOT, and CNO etc 2) Training time, inference speed, and memory usage are not compared for WDNO and baselines. 3) Lacks novelty as compared to previous works such as: * Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation (arXiv:2309.01745) * DiffusionPDE: Generative PDE-Solving Under Partial Observation (arXiv:2406.17763)
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications
MethodsDiffusion