An Informal Introduction to Multiplet Neural Networks

TL;DR

This paper introduces multiplet neural networks that replace the standard neuron with a group of neurons using generalized means, enabling the emulation of complex functions and enhancing learning dynamics.

Contribution

It proposes a novel multiplet neuron architecture based on Lehmer and Gini means, extending neural network capabilities and providing new insights into their properties and training.

Findings

Capable of solving XOR problem organically in two layers

Can perform multiplication and division operations

Can approximate functions using truncated power series

Abstract

In the artificial neuron, I replace the dot product with the weighted Lehmer mean, which may emulate different cases of a generalized mean. The single neuron instance is replaced by a multiplet of neurons which have the same averaging weights. A group of outputs feed forward, in lieu of the single scalar. The generalization parameter is typically set to a different value for each neuron in the multiplet. I further extend the concept to a multiplet taken from the Gini mean. Derivatives with respect to the weight parameters and with respect to the two generalization parameters are given. Some properties of the network are investigated, showing the capacity to emulate the classical exclusive-or problem organically in two layers and perform some multiplication and division. The network can instantiate truncated power series and variants, which can be used to approximate different functions, provided that parameters are constrained. Moreover, a mean case slope score is derived that can facilitate a learning-rate novelty based on homogeneity of the selected elements. The multiplet neuron equation provides a way to segment regularization timeframes and approaches.