# Realization theory of recurrent neural networks and rational systems

## TL;DR

This paper demonstrates that continuous-time recurrent neural networks can be represented by rational or polynomial systems under mild conditions, facilitating analysis of stability, identifiability, and minimality.

## Contribution

It introduces a method to embed RNNs into rational systems, enabling the application of realization theory to analyze RNN properties.

## Key findings

- Embedding RNNs into rational systems aids stability analysis.
- Provides algorithms for constructing polynomial and rational system representations.
- Derives conditions for RNN realizability and minimality.

## Abstract

In this paper, we show that, under mild assumptions, input-output behavior of a continous-time recurrent neural network (RNN) can be represented by a rational or polynomial nonlinear system. The assumptions concern the activation function of RNNs, and it is satisfied by many classical activation functions such as the hyperbolic tangent. We also present an algorithm for constructing the polynomial and rational system. This embedding of RNNs into rational systems can be useful for stability, identifiability, and realization theory for RNNs, as these problems have been studied for polynomial/rational systems. In particular, we use this embedding for deriving necessary conditions for realizability of an input-output map by RNN, and for deriving sufficient conditions for minimality of an RNN.