Approximation Capabilities of Neural Networks using Morphological Perceptrons and Generalizations

TL;DR

This paper investigates the approximation capabilities of neural networks using morphological perceptrons and their generalizations, revealing limitations in their universal approximation abilities compared to traditional and log-number system-based networks.

Contribution

It demonstrates that max-sum and related morphological neural networks lack universal approximation capabilities, contrasting them with log-number system implementations that do not.

Findings

Max-sum ANNs do not have universal approximation capabilities.

Signed-max-sum and max-star-sum variants also lack universality.

Log-number system implementations do have universal approximation capabilities.

Abstract

Standard artificial neural networks (ANNs) use sum-product or multiply-accumulate node operations with a memoryless nonlinear activation. These neural networks are known to have universal function approximation capabilities. Previously proposed morphological perceptrons use max-sum, in place of sum-product, node processing and have promising properties for circuit implementations. In this paper we show that these max-sum ANNs do not have universal approximation capabilities. Furthermore, we consider proposed signed-max-sum and max-star-sum generalizations of morphological ANNs and show that these variants also do not have universal approximation capabilities. We contrast these variations to log-number system (LNS) implementations which also avoid multiplications, but do exhibit universal approximation capabilities.