Extending the Universal Approximation Theorem for a Broad Class of Hypercomplex-Valued Neural Networks

TL;DR

This paper extends the universal approximation theorem to a broad class of hypercomplex-valued neural networks, including complex, quaternion, and tessarine networks, establishing their approximation capabilities.

Contribution

It introduces the concept of non-degenerate hypercomplex algebras and proves the universal approximation theorem for neural networks defined on these algebras.

Findings

Universal approximation holds for hypercomplex-valued neural networks on non-degenerate algebras.

The paper generalizes existing results to a broader class of hypercomplex systems.

It provides theoretical foundations for using hypercomplex neural networks in various applications.

Abstract

The universal approximation theorem asserts that a single hidden layer neural network approximates continuous functions with any desired precision on compact sets. As an existential result, the universal approximation theorem supports the use of neural networks for various applications, including regression and classification tasks. The universal approximation theorem is not limited to real-valued neural networks but also holds for complex, quaternion, tessarines, and Clifford-valued neural networks. This paper extends the universal approximation theorem for a broad class of hypercomplex-valued neural networks. Precisely, we first introduce the concept of non-degenerate hypercomplex algebra. Complex numbers, quaternions, and tessarines are examples of non-degenerate hypercomplex algebras. Then, we state the universal approximation theorem for hypercomplex-valued neural networks defined on a non-degenerate algebra.