PRDP: Progressively Refined Differentiable Physics
Kanishk Bhatia, Felix Koehler, Nils Thuerey
TL;DR
PRDP introduces a method to reduce computational costs in differentiable physics training by adaptively refining physics accuracy during training, achieving significant speedups without loss of accuracy.
Contribution
It proposes a novel approach that identifies the minimal physics refinement needed for effective neural network training, enabling adaptive and efficient differentiation.
Findings
Reduced training time by 62% in Navier-Stokes emulation
Effective across various differentiable physics scenarios
Applicable to both unrolled and implicit differentiation
Abstract
The physics solvers employed for neural network training are primarily iterative, and hence, differentiating through them introduces a severe computational burden as iterations grow large. Inspired by works in bilevel optimization, we show that full accuracy of the network is achievable through physics significantly coarser than fully converged solvers. We propose Progressively Refined Differentiable Physics (PRDP), an approach that identifies the level of physics refinement sufficient for full training accuracy. By beginning with coarse physics, adaptively refining it during training, and stopping refinement at the level adequate for training, it enables significant compute savings without sacrificing network accuracy. Our focus is on differentiating iterative linear solvers for sparsely discretized differential operators, which are fundamental to scientific computing. PRDP is…
Peer Reviews
Decision·ICLR 2025 Poster
This article is clearly written and introduces a promising solution to computational costs of training campaigns that include a linear PDE solver, end to end. The main result, that progressively increasing the accuracy of the solver in end-to-end training converges to similar performance as a fully converged solver while substantially reducing computing costs, is original. The benefit from incomplete convergence is very interesting. Although unexpected, the computational experiments show that th
The paper suffer weak baselines. The main use case of the method is for iterative solvers of large linear models, however, the authors use 1D and 2D examples which would likely be solved efficiently by a direct solver. The authors should at least provide one 3D example with the simple heat equation. The heat equation itself is also a weak baseline, more complex linear models such as Helmholtz equation could be considered. It is worrying that the benefit of the method diminish as the problem get
* The paper is well-written, with clear explanations of the intuitions and motivations behind the method. * The algorithm is straightforward and effectively delivers the intended results, as seen in the reduction of validation loss with the progressive refinement of the physics solver, particularly notable in Figure 4 for the 2D Heat and 2D Navier-Stokes cases. * The savings in training time and computational resources are substantial. * The appendix is thorough and well-organized, with especial
* Overall, the technical contribution of the paper is somewhat limited, with the main novelty being the proposed algorithm for iterative refinements. * All physics solvers employed rely on iterative linear solvers. It would have been interesting to see if the method also applies with other physics solvers. * With the exception of the final example (the neural-hybrid approach), the other examples appear to be simplified or illustrative cases without clear, concrete applications.
The topic this paper wants to tackle seems interesting. It seems intuitive that, considering the noiseness of neural network training and approximative nature of deep models, the physics solver does not need to fully converge for the network to achieve maximum possible accuracy. This paper proposes to use an adaptive strategy to progressively refine the physics solver and thus improve the training efficiency. Several experiments are conducted to verify the efficacy of the proposed method.
- This paper should provide more background information about *differentiable physics* to make readers better understand the core contribution of the proposed method. I am not an expert of this field, and I find this paper a little bit hard to follow, and also unaware of the broader context this paper lies in. - The experiment settings in this paper are not clearly presented. Considering that this is paper submitted to ICLR, I want to know what is the role of the neural networks in each experime
Code & Models
Videos
Taxonomy
TopicsModel Reduction and Neural Networks · Stochastic Gradient Optimization Techniques · Generative Adversarial Networks and Image Synthesis
MethodsFocus