About rectified sigmoid function for enhancing the accuracy of Physics-Informed Neural Networks

TL;DR

This paper introduces a rectified sigmoid activation function for neural networks to improve the accuracy of solving physical problems modeled by ODEs, demonstrating superior performance over traditional sigmoid functions.

Contribution

The paper proposes a novel rectified sigmoid activation function and associated algorithms for physics-informed neural networks, enhancing their accuracy in solving physical problems.

Findings

Neural networks with rectified sigmoid outperform those with sigmoid in accuracy.

Algorithms for data-driven initialization and neuron-by-neuron fitting are effective.

Numerical experiments confirm improved results on harmonic oscillator, relativistic slingshot, and Lorentz system.

Abstract

The article is devoted to the study of neural networks with one hidden layer and a modified activation function for solving physical problems. A rectified sigmoid activation function has been proposed to solve physical problems described by the ODE with neural networks. Algorithms for physics-informed data-driven initialization of a neural network and a neuron-by-neuron gradient-free fitting method have been presented for the neural network with this activation function. Numerical experiments demonstrate the superiority of neural networks with a rectified sigmoid function over neural networks with a sigmoid function in the accuracy of solving physical problems (harmonic oscillator, relativistic slingshot, and Lorentz system).