Artificial Intelligence for Dynamical Systems in Wireless Communications: Modeling for the Future

TL;DR

This paper reviews the application of dynamical systems theory to wireless communications over the past 30 years, emphasizing its potential for future research and the need to revisit this approach with current resources and demands.

Contribution

It provides a comprehensive overview of dynamical systems applications in wireless communications and advocates for renewed interest and research in this area.

Findings

Dynamical systems theory has been applied across all media layers in wireless communications.

Reservoir computing originated in wireless communications and should be further integrated.

Current resources and demands make the application of dynamical systems feasible and promising.

Abstract

Dynamical systems are no strangers in wireless communications. Our story will necessarily involve chaos, but not in the terms secure chaotic communications have introduced it: we will look for the chaos, complexity and dynamics that already exist in everyday wireless communications. We present a short overview of dynamical systems and chaos before focusing on the applications of dynamical systems theory to wireless communications in the past 30 years, ranging from the modeling on the physical layer to different kinds of self-similar traffic encountered all the way up to the network layer. The examples of past research and its implications are grouped and mapped onto the media layers of ISO OSI model to show just how ubiquitous dynamical systems theory can be and to trace the paths that may be taken now. When considering the future paths, we argue that the time has come for us to revive the interest in dynamical systems for wireless communications. It did not happen already because of the big question: can we afford observing systems of our interest as dynamical systems and what are the trade-offs? The answers to these questions are dynamical systems of its own: they change not only with the modeling context, but also with time. In the current moment the available resources allow such approach and the current demands ask for it. Reservoir computing, the major player in dynamical systems-related learning originated in wireless communications, and to wireless communications it should return.

Tables

Table 1. TABLE I: Examples of dynamical systems research from literature, reflecting continuous, but not intensive efforts in the area.

Reference	Description
Holliday et al. 2006, [4]	Tackling the open problem of capacity analysis of channels characterized as Markov chains by interpreting it in terms of Lyapunov exponents
Tannous et al. 1991, [5]	Low-order chaos in the multipath propagation channel with a strange attractor having a dimension between 4 and 5
Galdi et al. 2005, [6]	Accessible survey of ray chaos
Sirkeci-Mergen & Scaglione 2005, [7]	Continuum approximation of dense wireless networks, making a continuous distribution of nodes
Costamagna et al. 1994, [8]	First in a decade-spanning series of works suggesting a possible way of having an attractor from a known chaotic mapping to represent a channel error model. As the next order of approximation, one may use several attractors stitched together to get a behavior closer to the dynamics observed in experiments, and different channels might be represented with different system parameters.
Savkin et al. 2005, [9]	MAC for wireless networks as a hybrid system. The discrete flow of data (fast dynamics) in the network is approximated as a continuous process, while the status of nodes in the network (working/not working, slow dynamics) remains a discrete time variable.
Scutari et al. 2008, [10]	Applying tools from dynamical systems theory, stability theory and nonlinear control for multiuser MIMO (multiple input multiple output) systems
Moioli et al. 2020, [11]	Drawing inspiration from dynamical system inside human brain for sixth generation communication systems, and seeking ways to integrate those communication systems with the human brain

Full text

I Introduction

Everything is a dynamical system, depending on how you define everything (and how you define a dynamical system). In the realm of wireless communications, we can observe quantifiable system outputs evolving in time (an essential property of a dynamical system) at every level of the ISO model hierarchy, where some aspects of the behavior are seen as governed by simple rules and very ordered, while others are seemingly random. The random cases attract more attention as they are more difficult to predict and harness, and they usually include more interaction with the external factors: users and the environment. The question of whether to model these parts as dynamical systems is a multi-layered one.

Not all systems can be described by a low-order, low-complexity model. We could model the universe using elementary particles positions and momenta at a certain point in time, but the computational power for such a variant of Laplace’s demon surpasses the information limits of the same universe greatly. Measurements might be hard to obtain as well, and while throwing a die is a perfectly deterministic process with simple dynamics, not knowing the precise (and ever-changing) initial conditions at the time of the throw makes it look random (the level of understanding of the processes and the granularity of the model ties closely with the traditional understanding of uncertainty in engineering models [1, 2]). While randomness is usually portrayed as the opposite of determinism and modeled in a different fashion, dynamical systems embrace it as much as they embrace determinism. Random dynamical systems are a generalized version of the deterministic ones, as they allow for a stochastic component. In wireless communications, this is the channel equation we start with: a linear random dynamical system with noise as a random component.

The history of dynamical systems theory shows its early bond with wireless communications, as some of the fundamental dynamical systems theory concepts introduced by Poincare came from his wireless telegraphy seminars (1908). The development of both disciplines in the following decades continued going hand in hand with the oscillators that were the central object of interest in dynamical systems theory and an invaluable component of every radio device from their inception. The perspectives we investigate here are those opened in dynamical systems theory once the theory of deterministic chaos was established in the second half of the 20th century, with the notions of sensitive dependence on the initial conditions, fractal dimension attractors, ergodicity, etc. The path we chose is one of understanding the already existing dynamical phenomena within wireless systems and putting them to use.

The other road in observing dynamical systems in wireless communications is the one of chaotic communications. This has been the dominant topic in the area since the early nineties when the possibility of synchronization of two chaotic systems was demonstrated [3]. This topic has repeatedly been surveyed in the past and it represents the chaos added to a communication system, not the one found existing within it.

We begin our story by presenting the basics of dynamical systems theory. This will help us appreciate the efforts made in the past to identify elements of dynamical systems in wireless communications settings. These efforts will then be presented systematically, mapped onto the media layers of ISO OSI (International Organization for Standardization - Open Systems Interconnection) model to put the concepts into a context and to suggest the ways to proceed with the research today. Then we proceed with our central claim: the combination of dynamical systems theory and machine learning has a potential to radically change wireless communications performance. We offer some initial results motivated by recent developments in the field as a motivation for further work.

II Dynamical systems and chaos

While chaotic behavior remains the trademark of dynamical systems theory and the most interesting exhibit in its zoo, it also remains a rare catch. When we aim to understand a dynamical system, we are interested in its stability, periodicity, controllability, observability: properties of the system acting on its own and under our influence.

Giving attributes of a dynamical system to signal components previously considered to be random noise: (1) allows a better prediction, which in turn enables better suppression of interference; (2) opens an opportunity to examine it as “a feature, not a bug”- i.e. adds another degree of freedom; (3) offers a physical interpretation.

II-A Signals, Phase Space, Attractors

A dynamical system is, once we know all of its degrees of freedom and sources of dynamics, a system of differential or difference equations depending on whether we work in continuous or discrete time. However, we tend to know so much only about very simple models seen in nature, or the models we devise ourselves. Usually, a dynamical system seen in the wild is a black box for us.

Both the system of equations and a black box take inputs, change their states and produce outputs, which all change in time. The number of state variables of a system is its order, the order of the equations’ system in case we have a mathematical description. The outputs are usually some of the states of the system visible to us.

Traditionally, system identification, i.e. building a model from the limited knowledge about the black box, is a matter of statistics and special sets of test inputs. This is the way the wireless channel is estimated with a (pilot) signal. Often, we try to obtain static or linear dynamic models as they are easy to work with. However, they are usually valid only within a narrow time or parameter interval. Nonlinear systems ask for different identification and modeling methods.

Having $n$ state variables, it is often useful to plot them in an $n$-dimensional coordinate system, the phase space. This is where phase trajectories are observed; typical examples are shown in Figure 1. The unstable systems may diverge to infinity either quasi-periodically or aperiodically; the stable systems may converge to an equilibrium in the same manner; while the periodical systems remain confined to a cycle. However, the chaotic systems were found not to follow any of these patterns: they end up confined in a bounded part of state space called the attractor (unlike unstable systems), traverse it in a non-periodic manner (unlike periodic systems) and never converge to a single equilibrium (unlike stable systems). Often it is an example of a motion around two equilibria and jumping from orbiting one to orbiting the other, as seen in the celebrated Lorenz attractor shown in Fig. 1. The evolution of trajectories on the attractor demonstrates two important properties of a chaotic system. The first one is the sensitive dependence on initial conditions as two arbitrarily close phase space trajectories will separate exponentially fast on the attractor, rendering prediction of future motion impossible in the long run. The second one is ergodicity: a phase trajectory will get arbitrarily close to any point on the attractor, given enough time has passed.

The signals and the attractors are easily obtained when the mathematical model of the system exists: even though the nonlinear differential/difference equations governing the dynamics are usually not solvable in closed form, a numerical solution can be found. However, while the system is a black box, we usually have only few (typically, only one) system outputs available. How to reconstruct the other state variables? How many to reconstruct in the first place? The signal analysis and processing for chaotic dynamical systems has a toolbox for this task. While a detailed description goes out of the scope of this paper, Fig. 2 gives an overview of the attractor reconstruction and quantitative analysis of the results.

II-B The Metrics

Dynamical systems are quantitative, so many metrics are devised to assess and categorize them. From the viewpoint of control and stability, measures of stability margins describe how stable and robust a system is and how much disturbance it could take without a failure.

For chaotic systems, the measure called the Lyapunov exponent gained importance. In a conventional, non-chaotic deterministic system, it is a negative exponent which describes how fast two separate phase trajectories converge. In chaotic systems, however, we have already learned that even infinitesimally close trajectories diverge at an exponential rate: a positive Lyapunov exponent describes this dynamic. Positive Lyapunov exponent thus became a symbol of chaos and a basis for its measure.

An $n$-dimensional dynamical system has $n$ Lyapunov exponents, and if it is chaotic, at least one of them is positive, resulting in sensitive dependence on the initial conditions. If at least two Lyapunov exponents are positive, we speak of hyper-chaotic systems. Now, how do we measure just how chaotic a system is, and how do we compare two chaotic systems in any way?

One possible entropy-based approach, the Kolmogorov-Sinai entropy is related to the positive Lyapunov exponents and may give us a hint of just how unpredictable a chaotic system is, with a numerical value between 0 (non-chaotic deterministic systems) and infinity (purely random systems). The link with Shannon’s entropy and information theory in general is straightforward, but interesting non-trivial results linking Lyapunov exponents of random dynamical systems and entropy in wireless channels suggest there is more to it than it meets the eye [4].

Another metric is the dimension. In the usual sense, we perceive dimension of a geometric construct as an integer, living in a 3D space, observing images as 2D projections, etc. However, if we make a finer measure of the dimension (e.g. Hausdorff dimension), to describe just how much of space the object whose dimension we measure takes, we discover that not everything is integer-dimensional. In particular, observing the Lorenz attractor in Figure 1 may lead to a conclusion that it is not exactly 2-dimensional: its dimension is in fact just slightly larger than two. The non-integer dimension of strange attractors is thus another indicator and a measure of chaos.

When speaking of non-integer dimensions, it is necessary to mention self-similarity and fractals, as one of the most often mentioned features in dynamical systems and chaos, at least in the popular view. Fractals are self-similar structures, as a zoomed in part of it looks just like the bigger one, iterating the structure. The infinitely rough structure of a fractal results in its non-integer Hausdorff dimension, while retaining integer topological dimension (e.g. fractals with the topological dimension of 2 can fit in a plane).

III Up and Down the Media Layers

The quest for chaos and other interesting dynamical system properties was a hot topic with all features of a bandwagon at the end of the last century. The development of algorithms described in the previous section to determine chaoticity of the data (and the systems generating it) brought a series of investigations and results in different areas of science and engineering. Chaos was looked for in the phenomena previously considered random (from the stock market to temperature oscillations), and dynamical system formulation was sought in the fields where the systems were not considered quantitative at all, such as learning processes and interpersonal interaction.

Wireless communications followed suit, and the aspects examined were all over the media layers of the ISO OSI model describing it. The logic behind is simple: where there is a time series or a signal to be measured, there is a dynamical system producing it. And as we have seen, we can find out the details about the system from the outputs, even more so if it is chaotic. A preview of this historical review is given in Figure 3 and elaborated upon in this section. Illustrative references related to different layers are presented in Table I.

III-A The Physics

The wireless channel has been viewed as a dynamical system from the early beginning, but keeping a lot of its effect on the signal under the umbrella of random noise and unpredictable changes. Chaos-theoretic tools and new trends in dynamical systems theory have provided some hope to distinguish the magic of noise from the science of complexity and chaos in the channel.

Detecting low order chaos in multipath propagation in the early nineties [5] opened some promising paths for the future, but similar experiments performed later show that the deterministic chaos is not always present. The focus turned to sea clutter in radar context, but again the results have been inconclusive, and again coming short of the revolutionary applications chaotic sea clutter nature might have in detection and tracking of targets near the sea surface, if and when confirmed [12].

For a moment, examine the billiard ball system in which a tiny difference in the initial condition drastically changes the trajectory of the billiard ball, depending on the table shape and the position of other balls. The billiard ball analogy works to an extent in the context of electromagnetic waves in a chamber and explains ray chaos: the divergence of two electromagnetic waves originating from almost the same place, due to multiple bounces off the environment [6]. Ray chaos is achievable in other wave settings as well, e.g. the acoustic case. The case of sound waves is actually the one from which we borrow a useful application of ray chaos for radio: enabling good time reversal.

Time reversal in acoustics is the idea of creating a sound wave which is a time-reversed version of a given sound wave received: like an echo, but converging into the point of the original wave’s source. This idea [13] extends to optics and radio communications, serving as the core principle of several localization and communication schemes, e.g. conjugate beamforming. In a rich scattering environment with a lot of reflections, the bounces in the multipath quickly map the entire space and allow good time reversal. A chaotic environment has this property as well, as it exhibits ergodicity: the system will eventually pass through any part of its attractor, if we can afford to wait.

Time in dynamical system can either be a discrete or a continuous variable. An interesting example of blurring the lines between the two is the calculation of a continuous-time limit of a discrete system. The continuous model is a limit of a discrete model as the discrete time interval shrinks. If the systems are spatial instead of temporal (i.e. a distance plays the role of time), the shrinking distance produces a continuum in the limit process. In a network, the infinitely increasing density aims for the limiting continuous process [7] (Table I). The resulting continuous system may either be an approximation good enough, or a way to obtain lower/upper bounds on the discrete system performance.

III-B Data Link Layer

Taking a step up on the ladder, we stay in the realm of channel modeling. The question of channel errors, their occurrence, modeling and distribution is observed and answered at the data link layer [8]. Channel error models are dynamical systems: they have a time evolution, have outputs in the form of discrete time series, but are they chaotic? Can a model based on chaotic systems encompass the dynamics of channel errors?

The subtle problem of modeling is bringing in all the existing elements of the real system into the model while keeping out all the non-existing ones. A certain dynamical system model that happens to be chaotic may represent certain statistical properties of the real system well. However, if the original system is not chaotic, the model is bringing a lot of rich, but unwanted properties of its own.

The example of channel errors is an example of a discrete-time system, but the processes we describe at this layer could be inherently a combination of both continuous and discrete time modes. This is the case of a discrete system interacting with a continuous one, one being embedded within the other. Medium access control (MAC) for wireless networks can be modeled as such a system [9] (Table I).

III-C *Self-similar traffic across the layers *

The characteristics of the traffic in (wireless) networks have been investigated for years from the perspective of signal processing, statistics, linear and nonlinear signal theory. The identification of statistical properties and nonlinear model parameters of both wireless LAN (local area network) and IP (internet protocol) traffic would enable better models and therefore better control, prediction, caching and routing. An elementary question is that of memory: is the time series of the wireless LAN traffic showing any long range dependence over time, or it is just coin tossing at every time instant? The same question was raised at the network layer for the IP traffic, observing the TCP (transmission control protocol) congestion control. The rise and the fall of the Nile, a celebrated example of complex patterns in nature is an example of a system’s memory. In the study of the Nile, Hurst introduced the notion of Hurst Exponent as a measure of long term dependence in data. It was not too surprising when the Hurst exponents were found to be directly related with fractal dimension of self-similar data, as the memory of the system described by the Hurst exponent is affecting the smoothness of a signal and the fractal scale.

The wireless LAN traffic was expected to be self-similar: in the mid-nineties, the self-similarity in Ethernet traffic was detected and the research community had a field day in wireless traffic. The packets, the bursts, the round trip time, the errors, self-similarity appeared to be everywhere. A contributing factor for this illusion was the non-existence of the one right way to determine and quantify self-similarity in data. Another reason was the existence of external effects creating an illusion of self similarity, such as periodic interference. The case of IP traffic was more fruitful in self-similarity terms, but less relevant to the wireless context.

Down on the physical layer, some researchers have suggested the introduction of generated self-similar signals into wireless communication. One example is the use of self-similar carriers (and consequently, fractal modulation), which had limited adoption [14]. The physical fractal structure, however, has a long tradition in antenna design for wireless communications. A typical fractal antenna has a fractal shape and puts to use the two advantages of fractals, the existence of scaled structures and the space-filling properties. The self-similar scaled structures offer different scales of length for the antenna to work at, directly resulting in similar effects for different wavelengths. This does not necessarily mean good performance for the said wavelengths, so the story of fractal antennas is not that straightforward. The space-filling property is related to the capability of all fractals to achieve dense packing in some parts (in terms of antennas, it is dense packing of wires within a small surface area). For chaotic signals, it was suggested that their transmission would be robust to multipath effects and severe noise [15].

III-D The Cameos: brain, control and games

Wireless communications have a long term relationship with several disciplines heavily relying on dynamical systems. This part of the story must not be overlooked, so we examine the control over wireless and the game theory in the wireless setting.

First, we note the cameo role of dynamical systems in wireless communication as seen in differential games. Defined as a game over a dynamical system, a differential game can model, control and optimize various processes in wireless communications. Again, it is very often a hybrid dynamical system the control is performed on, dealing with external dynamics such as drone or robotic movement, but also the intrinsic “dynamics” of wireless including power control, transmission rates and delays [10]. To make it realistic and useful for practical considerations, the models of the dynamical systems have to be faithful to the reality. This means that the differential game theory eagerly awaits the results of everything dynamical systems research can get from the wireless of the day.

Closely related to game theory is the control over wireless: it uses the wireless channel as the control signal medium and has to deal with all of its peculiarities, determinism and randomness alike (a timely example is that of teleoperation [16]). The control engineers got rid of the wires cluttering the factory and decreasing mobility, but had to face a whole new world of wireless communications, technologies and protocols. The distributed control system just got another dynamical system on top of it, between its nodes: the wireless. The ubiquitous wireless sensor networks are essentially control networks, just without actuators. And there again the dynamical systems emerge: the ones whose outputs are measured by the sensors and the ones the sensory data travels through.

Finally, one area of wireless communications that recognises the importance of dynamical systems and chaos is at the crossroads of engineering and neuroscience, where researchers both draw inspiration from the dynamical systems and chaotic activity in brain for bio-inspired network and device design, and seek ways to integrate modern wireless communications with human organism (brain in particular), for health-related applications of the next generation communications [11].

IV Learning Dynamical Systems in Wireless Communications

The two decades of the new century saw the new artificial intelligence spring, growing data availability and the growing capacity for data handling. In wireless communications, an early major milestone was the pioneering work on reservoir computing [17]. This concept, illustrated in Fig. 4 is the quintessential machine learning model for dynamical systems: the intermediate stage between inputs and outputs in this network is a dynamical system on its own–it does not attempt to adapt to the actual system, except for the subset of connections leading to the outputs. As such a general dynamical systems tool, reservoir computing quickly left the realm of wireless communication (only two reservoir computing applications cited in the most recent surveys of learning in this area [18, 19]) and found greener pastures. It aims, together with other machine learning paradigms, at providing a helping hand in predicting the behavior of otherwise hard-to-anticipate nonlinear systems, but we are still arguably waiting for revolutionary results. They are within the reach once we offer a helping hand to the machine learning as well: we need better models of dynamical systems for it to work on. More detailed models may ask for more computing power, but we do have it now, and in turn they greatly reduce the search space for the machine learning and allow it to focus its efforts.

Recent developments suggest that even the black box approach in which the model has no knowledge of the actual physicality of the process has a lot to offer for nonlinear dynamic systems. The application of reservoir computing to spatiotemporal chaotic systems allowed an expansion of prediction horizon–as we stated earlier, prediction of chaotic behavior is hard as two infinitesimaly close trajectories diverge exponentially fast, and they separate within the time period called *Lyapunov time *(cf. Lyapunov exponents). Pathak et al. [20] report the extension of reliable prediction window from one Lyapunov time interval to eight Lyapunov times for a particular chaotic system (Kuramoto-Sivashinsky equation). This promising result, applicable to wider classes of chaotic systems, motivates our investigation of its effects in wireless communications: how to convert the information about the future into gains in basic communication quality parameters such as sum rates and latency? Improvement in these metrics has been a significant driver of the technological progress, a major argument for inclusion of new approaches (massive MIMO, mmWave) in the new standards, and a defining aspect of the current wireless communications generation, the 5G. In this analysis, we focus on sum rate increase, *computational burden decrease *and latency decrease (Fig. 5(a)). These results from simple use case scenarios (Fig. 5(b-d)) suggest significant direct benefit from dynamical systems approach to system state prediction and add quantitative incentives to our initiative for dynamical systems research: even relatively short prediction has a major effect on the observed variable, justifying investing resources into it.

The scenarios are inspired by use cases on next generation networks. The co-located massive MIMO example (5(b)) explores the question of what benefit for sum rates can we observe if we can predict changes in channel state information (CSI) between coherence intervals (periods in which CSI can be considered known and constant). The distributed massive MIMO example (5(c)) was used to compare the computational overhead for prediction against the computation needed for sensing, processing, and re-calculation of CSI updates. Finally, the mobile multi-hop system example (5(d)) was set up inspired by traffic scenarios (e.g. autonomous cars in urban settings) to see how predictability of motion helps in minimising the message delivery time. These examples and their spatiotemporal variability link well with the original work of Pathak et al. [20] .

When we speak of models, their integration with machine learning, and reservoir computing, it is interesting to note that Pecora and Carroll, the same researchers who have founded the field of chaotic communications with their work on synchronisation of circuits [3], have also recently done major work on understanding the effect choice of network connections within the reservoir [21]. Depending on the dynamical system the reservoir computer is trying to predict the behaviour of, the structure of computer itself does in fact matter: flipping some of the fixed connections in the reservoir (full line arrows in Fig. 4) deliver different prediction results for the same inputs. Design of structures, given information about application domain, hence becomes an important segment of work and asks for understanding of complex networks. Network science and study of complex networks are additional promising techniques for the telecommunications community’s emergent toolbox of the future. The complexity is both a blessing and a curse: unintended consequences may lead to massive failures of complex networks of ”smart agents” [22], but carefully designed solutions have a lot of promise for the future of technology facing climate emergency and the demise of current economic systems [23].

V Conclusions

The time is now: the toolbox for dynamical systems has been reinforced with machine learning techniques born out of dynamical systems, and the dynamics of wireless communications offer much more to work with every day. This does not mean that the work done in the past was not important; we need to revisit it with techniques and technology we have today, and the results might have an application that could not have been foreseen decades ago.

Again, to repeat the statement we began with: the question is not whether we can treat everything in a wireless network as a dynamical system, but whether we can afford to do so. The demand is high, as the 6G and the generations of wireless to follow will benefit from getting to know the nature of the complex, dynamical world they are creating and embedding into at the same time. It is hard to find a use case of next generation networks where we could not see a nonlinear differential equation waiting to be modeled: be it the “wetware” integration with dynamics of a human body, motion in the high mobility scenario, or the myriad of rays running into potentially ray-chaotic states in Massive MIMO, terahertz communications, metasurfaces, the dynamical systems are within reach.