Time ordering in the evolution of information processing and modulation systems

TL;DR

This paper reviews optimization in neural networks, proposes a simple model for information processing system development, and explores potential biological relevance of these ideas across different levels of organization.

Contribution

It introduces a model of subsidiary variables and time ordering in learning systems, linking artificial and biological information processing evolution.

Findings

Proposes a simple model for improving learning in feedforward networks.

Suggests time ordering of information processing systems in evolution.

Indicates potential applicability to biological systems across scales.

Abstract

The ideas of optimization of learning algorithms in Artificial Neural Networks are reviewed emphasizing generic properties and the online implementations are interpreted from a biological perspective. A simple model of the relevant subsidiary variables needed to improve learning in artificial feedforward networks and the `time ordering' of the appearance of the respective information processing systems is proposed. We discuss the possibility that these results might be relevant in other contexts, not being restricted to the simple models from which they stem. The analysis of a few examples, which range from the lowest cellular scale to the macroscopic level, suggests that similar ideas could be applied to biological systems.