Collaboration between parallel connected neural networks -- A possible criterion for distinguishing artificial neural networks from natural organs

TL;DR

This paper experimentally investigates properties of parallel-connected neural networks, revealing unique behaviors that distinguish artificial networks from biological organs and proposing a criterion for measuring their bionic level.

Contribution

The study introduces a novel criterion based on parallel neural network properties to differentiate artificial from natural neural systems and evaluates activation functions' bionic qualities.

Findings

Parallel-connected neural networks can output correct results even when all sub-networks fail.

Sub-networks in a PNN are not necessarily optimized individually.

ReLU activation function enhances the bionic level of neural networks.

Abstract

We find experimentally that when artificial neural networks are connected in parallel and trained together, they display the following properties. (i) When the parallel-connected neural network (PNN) is optimized, each sub-network in the connection is not optimized. (ii) The contribution of an inferior sub-network to the whole PNN can be on par with that of the superior sub-network. (iii) The PNN can output the correct result even when all sub-networks give incorrect results. These properties are unlikely for natural biological sense organs. Therefore, they could serve as a simple yet effective criterion for measuring the bionic level of neural networks. With this criterion, we further show that when serving as the activation function, the ReLU function can make an artificial neural network more bionic than the sigmoid and Tanh functions do.