Spatio-Temporal Activation Function To Map Complex Dynamical Systems

TL;DR

This paper introduces a novel spatio-temporal activation function that enables neural networks to model complex and chaotic dynamical systems more effectively without using recurrent architectures.

Contribution

A new two-dimensional activation function with an added temporal component is proposed to capture complex dynamics in time series data.

Findings

The activation function successfully models chaotic systems.

It reduces reliance on recurrent neural networks.

Demonstrates improved dynamic behavior modeling.

Abstract

Most of the real world is governed by complex and chaotic dynamical systems. All of these dynamical systems pose a challenge in modelling them using neural networks. Currently, reservoir computing, which is a subset of recurrent neural networks, is actively used to simulate complex dynamical systems. In this work, a two dimensional activation function is proposed which includes an additional temporal term to impart dynamic behaviour on its output. The inclusion of a temporal term alters the fundamental nature of an activation function, it provides capability to capture the complex dynamics of time series data without relying on recurrent neural networks.