Feedforward Neural Networks with Diffused Nonlinear Weight Functions

TL;DR

This paper introduces feedforward neural networks with nonlinear weight functions smoothed via diffusion, aiming to reduce the number of parameters and improve performance while maintaining computational efficiency.

Contribution

It proposes a novel neural network architecture with diffused nonlinear weight functions, potentially reducing the total number of weights needed for learning.

Findings

Networks perform better than classic neural networks in some cases

Diffused nonlinear weight functions can improve learning efficiency

Maintains low computational complexity during propagation

Abstract

In this paper, feedforward neural networks are presented that have nonlinear weight functions based on look--up tables, that are specially smoothed in a regularization called the diffusion. The idea of such a type of networks is based on the hypothesis that the greater number of adaptive parameters per a weight function might reduce the total number of the weight functions needed to solve a given problem. Then, if the computational complexity of a propagation through a single such a weight function would be kept low, then the introduced neural networks might possibly be relatively fast. A number of tests is performed, showing that the presented neural networks may indeed perform better in some cases than the classic neural networks and a number of other learning machines.