VS-PINN: A fast and efficient training of physics-informed neural networks using variable-scaling methods for solving PDEs with stiff behavior

TL;DR

This paper introduces VS-PINN, a variable-scaling method that enhances the training efficiency and accuracy of physics-informed neural networks for solving stiff PDEs with high-frequency solutions, supported by theoretical and experimental evidence.

Contribution

The paper presents a novel variable-scaling technique for PINNs that improves training speed and performance on stiff PDEs, backed by NTK analysis.

Findings

Significant improvement in training efficiency for stiff PDEs

Enhanced accuracy in high-frequency solution problems

Theoretical validation via neural tangent kernel analysis

Abstract

Physics-informed neural networks (PINNs) have recently emerged as a promising way to compute the solutions of partial differential equations (PDEs) using deep neural networks. However, despite their significant success in various fields, it remains unclear in many aspects how to effectively train PINNs if the solutions of PDEs exhibit stiff behaviors or high frequencies. In this paper, we propose a new method for training PINNs using variable-scaling techniques. This method is simple and it can be applied to a wide range of problems including PDEs with rapidly-varying solutions. Throughout various numerical experiments, we will demonstrate the effectiveness of the proposed method for these problems and confirm that it can significantly improve the training efficiency and performance of PINNs. Furthermore, based on the analysis of the neural tangent kernel (NTK), we will provide theoretical evidence for this phenomenon and show that our methods can indeed improve the performance of PINNs.