Multilayer Perceptron-based Surrogate Models for Finite Element Analysis

TL;DR

This paper investigates how different hyperparameters, such as activation functions and learning rates, affect the performance of Physics-Informed Neural Networks in solving PDEs, highlighting GELU and tanh as superior choices.

Contribution

It provides a systematic analysis of hyperparameter effects on PINN performance, offering guidance for optimal configurations in PDE solutions.

Findings

GELU and tanh outperform other activation functions.

Proper learning rate selection improves accuracy and convergence speed.

Hyperparameter tuning enhances PINN effectiveness for PDEs.

Abstract

Many Partial Differential Equations (PDEs) do not have analytical solution, and can only be solved by numerical methods. In this context, Physics-Informed Neural Networks (PINN) have become important in the last decades, since it uses a neural network and physical conditions to approximate any functions. This paper focuses on hypertuning of a PINN, used to solve a PDE. The behavior of the approximated solution when we change the learning rate or the activation function (sigmoid, hyperbolic tangent, GELU, ReLU and ELU) is here analyzed. A comparative study is done to determine the best characteristics in the problem, as well as to find a learning rate that allows fast and satisfactory learning. GELU and hyperbolic tangent activation functions exhibit better performance than other activation functions. A suitable choice of the learning rate results in higher accuracy and faster convergence.