Approximations with deep neural networks in Sobolev time-space

TL;DR

This paper develops a framework demonstrating that deep neural networks, specifically with ReCU activation, can effectively approximate functions in Sobolev spaces relevant to evolution equations, addressing regularity and integrability in time and space.

Contribution

The paper introduces a novel approach using ReCU activation functions to approximate Sobolev-regular functions in Bochner-Sobolev spaces, overcoming limitations of ReLU.

Findings

Neural networks with ReCU can approximate Sobolev functions effectively.

ReCU activation avoids issues caused by non-regularity of ReLU.

Framework applicable to evolution equations in Sobolev spaces.

Abstract

Solutions of evolution equation generally lies in certain Bochner-Sobolev spaces, in which the solution may has regularity and integrability properties for the time variable that can be different for the space variables. Therefore, in this paper, we develop a framework shows that deep neural networks can approximate Sobolev-regular functions with respect to Bochner-Sobolev spaces. In our work we use the so-called Rectified Cubic Unit (ReCU) as an activation function in our networks, which allows us to deduce approximation results of the neural networks while avoiding issues caused by the non regularity of the most commonly used Rectivied Linear Unit (ReLU) activation function.