Fully tensorial approach to hypercomplex-valued neural networks

TL;DR

This paper introduces a comprehensive tensorial framework for hypercomplex-valued neural networks, enabling algebraic operations to be expressed through tensor contractions, thus unifying and generalizing existing models across arbitrary finite-dimensional algebras.

Contribution

It develops a novel tensor-based approach to hypercomplex neural networks that is compatible with modern deep learning tools and extends the universal approximation theorem to this setting.

Findings

Unified tensorial formulation for hypercomplex layers

Compatibility with deep learning libraries

Universal approximation theorem established

Abstract

A fully tensorial theoretical framework for hypercomplex-valued neural networks is presented. The proposed approach enables neural network architectures to operate on data defined over arbitrary finite-dimensional algebras. The central observation is that algebra multiplication can be represented by a rank-three tensor, which allows all algebraic operations in neural network layers to be formulated in terms of standard tensor contractions, permutations, and reshaping operations. This tensor-based formulation provides a unified and dimension-independent description of hypercomplex-valued dense and convolutional layers and is directly compatible with modern deep learning libraries supporting optimized tensor operations. The proposed framework recovers existing constructions for four-dimensional algebras as a special case. Within this setting, a tensor-based version of the universal approximation theorem for single-layer hypercomplex-valued perceptrons is established under mild non-degeneracy assumptions on the underlying algebra, thereby providing a rigorous theoretical foundation for the considered class of neural networks.