A Modified Sigma-Pi-Sigma Neural Network with Adaptive Choice of Multinomials

TL;DR

This paper introduces a modified Sigma-Pi-Sigma neural network that adaptively selects better multinomials for improved mapping capabilities, outperforming traditional SPSNNs on benchmark problems.

Contribution

It proposes an adaptive approach to select multinomials in SPSNNs using regularization, enhancing their flexibility and performance.

Findings

MSPSNN outperforms traditional SPSNN on benchmark problems.

Adaptive multinomial selection improves neural network mapping capability.

Regularization effectively reduces unnecessary monomials.

Abstract

Sigma-Pi-Sigma neural networks (SPSNNs) as a kind of high-order neural networks can provide more powerful mapping capability than the traditional feedforward neural networks (Sigma-Sigma neural networks). In the existing literature, in order to reduce the number of the Pi nodes in the Pi layer, a special multinomial P_s is used in SPSNNs. Each monomial in P_s is linear with respect to each particular variable sigma_i when the other variables are taken as constants. Therefore, the monomials like sigma_i^n or sigma_i^n sigma_j with n>1 are not included. This choice may be somehow intuitive, but is not necessarily the best. We propose in this paper a modified Sigma-Pi-Sigma neural network (MSPSNN) with an adaptive approach to find a better multinomial for a given problem. To elaborate, we start from a complete multinomial with a given order. Then we employ a regularization technique in the learning process for the given problem to reduce the number of monomials used in the multinomial, and end up with a new SPSNN involving the same number of monomials (= the number of nodes in the Pi-layer) as in P_s. Numerical experiments on some benchmark problems show that our MSPSNN behaves better than the traditional SPSNN with P_s.