Positive Neural Networks in Discrete Time Implement Monotone-Regular Behaviors

TL;DR

This paper explores the capabilities of positive neural networks in discrete time, demonstrating they can implement all monotone-regular behaviors with certain delays, highlighting their expressive power in modeling regular language-based behaviors.

Contribution

It establishes that positive neural networks can realize all monotone-regular behaviors with a delay of one time unit, revealing their expressive power in discrete-time settings.

Findings

Positive neural networks capture all monotone-regular behaviors.

Some behaviors can be implemented with zero delay.

Certain simple behaviors cannot be implemented with zero delay.

Abstract

We study the expressive power of positive neural networks. The model uses positive connection weights and multiple input neurons. Different behaviors can be expressed by varying the connection weights. We show that in discrete time, and in absence of noise, the class of positive neural networks captures the so-called monotone-regular behaviors, that are based on regular languages. A finer picture emerges if one takes into account the delay by which a monotone-regular behavior is implemented. Each monotone-regular behavior can be implemented by a positive neural network with a delay of one time unit. Some monotone-regular behaviors can be implemented with zero delay. And, interestingly, some simple monotone-regular behaviors can not be implemented with zero delay.