Quantitative approximation results for complex-valued neural networks

TL;DR

This paper provides explicit quantitative error bounds for approximating smooth functions using complex-valued neural networks with modReLU activation, advancing the theoretical understanding of their expressivity.

Contribution

It offers the first explicit approximation error bounds for complex neural networks with modReLU, demonstrating their optimal approximation rates for smooth functions.

Findings

Derived explicit error bounds for complex neural network approximation.

Showed the approximation rates are optimal up to log factors.

Extended the theoretical understanding of complex-valued neural networks.

Abstract

Until recently, applications of neural networks in machine learning have almost exclusively relied on real-valued networks. It was recently observed, however, that complex-valued neural networks (CVNNs) exhibit superior performance in applications in which the input is naturally complex-valued, such as MRI fingerprinting. While the mathematical theory of real-valued networks has, by now, reached some level of maturity, this is far from true for complex-valued networks. In this paper, we analyze the expressivity of complex-valued networks by providing explicit quantitative error bounds for approximating $C^n$ functions on compact subsets of $\mathbb{C}^d$ by complex-valued neural networks that employ the modReLU activation function, given by $\sigma(z) = \mathrm{ReLU}(|z| - 1) \, \mathrm{sgn} (z)$, which is one of the most popular complex activation functions used in practice. We show that the derived approximation rates are optimal (up to log factors) in the class of modReLU networks with weights of moderate growth.