Neural Networks and Continuous Time

TL;DR

This paper emphasizes the importance of neural networks designed for continuous time to model ongoing processes, enabling better representation of dynamic behaviors and periodic phenomena beyond discrete functions.

Contribution

It introduces the concept of continuous-time neural networks and relates them to automata models for representing continuous and hybrid processes.

Findings

Continuous-time neural networks enable modeling of ongoing processes.

They facilitate analysis of periodic behaviors in neural systems.

The paper connects neural architectures with automata models for continuous dynamics.

Abstract

The fields of neural computation and artificial neural networks have developed much in the last decades. Most of the works in these fields focus on implementing and/or learning discrete functions or behavior. However, technical, physical, and also cognitive processes evolve continuously in time. This cannot be described directly with standard architectures of artificial neural networks such as multi-layer feed-forward perceptrons. Therefore, in this paper, we will argue that neural networks modeling continuous time are needed explicitly for this purpose, because with them the synthesis and analysis of continuous and possibly periodic processes in time are possible (e.g. for robot behavior) besides computing discrete classification functions (e.g. for logical reasoning). We will relate possible neural network architectures with (hybrid) automata models that allow to express continuous processes.