Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation

TL;DR

This paper introduces an evolutionary computation approach to automatically discover physics-informed neural network models that achieve higher accuracy and faster convergence in solving PDEs, surpassing traditional hyperparameter optimization methods.

Contribution

The study presents a novel evolutionary algorithm that optimizes neural network architecture, shortcut connections, and activation functions for PINNs, enhancing PDE solving performance.

Findings

Models discovered by the method outperform Bayesian optimization and random search in accuracy.

The evolutionary approach yields models with faster convergence rates.

Discovered models demonstrate strong generalization across different PDE conditions.

Abstract

In recent years, the researches about solving partial differential equations (PDEs) based on artificial neural network have attracted considerable attention. In these researches, the neural network models are usually designed depend on human experience or trial and error. Despite the emergence of several model searching methods, these methods primarily concentrate on optimizing the hyperparameters of fully connected neural network model based on the framework of physics-informed neural networks (PINNs), and the corresponding search spaces are relatively restricted, thereby limiting the exploration of superior models. This article proposes an evolutionary computation method aimed at discovering the PINNs model with higher approximation accuracy and faster convergence rate. In addition to searching the numbers of layers and neurons per hidden layer, this method concurrently explores the optimal shortcut connections between the layers and the novel parametric activation functions expressed by the binary trees. In evolution, the strategy about dynamic population size and training epochs (DPSTE) is adopted, which significantly increases the number of models to be explored and facilitates the discovery of models with fast convergence rate. In experiments, the performance of different models that are searched through Bayesian optimization, random search and evolution is compared in solving Klein-Gordon, Burgers, and Lam\'e equations. The experimental results affirm that the models discovered by the proposed evolutionary computation method generally exhibit superior approximation accuracy and convergence rate, and these models also show commendable generalization performance with respect to the source term, initial and boundary conditions, equation coefficient and computational domain. The corresponding code is available at https://github.com/MathBon/Discover-PINNs-Model.