Artificial Neurons with Arbitrarily Complex Internal Structures

TL;DR

This paper introduces a new class of artificial neurons with complex internal structures, demonstrating their enhanced computational capacity and establishing a link to standard neural network architectures.

Contribution

It presents a generalized neuron model with complex internal variables and functions, showing increased information capacity and a connection to three-layer feed-forward networks.

Findings

Maximum information capacity of attractor networks reaches theoretical bounds.

Generalized neurons can model complex internal dynamics.

A correspondence between generalized attractor networks and standard feed-forward networks is established.

Abstract

Artificial neurons with arbitrarily complex internal structure are introduced. The neurons can be described in terms of a set of internal variables, a set activation functions which describe the time evolution of these variables and a set of characteristic functions which control how the neurons interact with one another. The information capacity of attractor networks composed of these generalized neurons is shown to reach the maximum allowed bound. A simple example taken from the domain of pattern recognition demonstrates the increased computational power of these neurons. Furthermore, a specific class of generalized neurons gives rise to a simple transformation relating attractor networks of generalized neurons to standard three layer feed-forward networks. Given this correspondence, we conjecture that the maximum information capacity of a three layer feed-forward network is 2 bits per weight.