Stability of Algebraic Neural Networks to Small Perturbations

TL;DR

This paper investigates the stability of algebraic neural networks (AlgNNs) within algebraic signal processing, demonstrating that architectures based on formal convolution can be stable regardless of the shift operator, depending on algebraic structure.

Contribution

It provides a theoretical framework showing stability conditions for AlgNNs using algebraic signal models, especially for algebras with a single generator.

Findings

AlgNNs stability depends on algebraic structure.

Stability holds beyond specific shift operators.

Focus on algebras with a single generator.

Abstract

Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model, and information is mapped between layers by means of a nonlinearity function. AlgNNs provide a generalization of neural network architectures where formal convolution operators are used, like for instance traditional neural networks (CNNs) and graph neural networks (GNNs). In this paper we study stability of AlgNNs on the framework of algebraic signal processing. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator, and this stability depends on the structure of subsets of the algebra involved in the model. We focus our attention on the case of algebras with a single generator.