Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions

TL;DR

This paper enhances the extrapolation ability of Physics-Informed Neural Networks by employing transfer learning within an extended domain and introducing an adaptive activation function, resulting in significantly improved accuracy with minimal additional computational cost.

Contribution

It introduces a transfer learning approach and an adaptive activation function to improve PINN extrapolation performance and robustness.

Findings

40% reduction in relative L2 error on average

50% reduction in mean absolute error

Improved robustness and accuracy without significant computational cost

Abstract

Physics-Informed Neural Networks (PINNs) are deep learning models that incorporate the governing physical laws of a system into the learning process, making them well-suited for solving complex scientific and engineering problems. Recently, PINNs have gained widespread attention as a powerful framework for combining physical principles with data-driven modeling to improve prediction accuracy. Despite their successes, however, PINNs often exhibit poor extrapolation performance outside the training domain and are highly sensitive to the choice of activation functions (AFs). In this paper, we introduce a transfer learning (TL) method to improve the extrapolation capability of PINNs. Our approach applies transfer learning (TL) within an extended training domain, using only a small number of carefully selected collocation points. Additionally, we propose an adaptive AF that takes the form of a linear combination of standard AFs, which improves both the robustness and accuracy of the model. Through a series of experiments, we demonstrate that our method achieves an average of 40% reduction in relative L2 error and an average of 50% reduction in mean absolute error in the extrapolation domain, all without a significant increase in computational cost. The code is available at https://github.com/LiuzLab/PINN-extrapolation .