Averaging plus Learning Models and Their Asymptotics

TL;DR

This paper introduces new models for social learning in financial and social networks, analyzing how agents' beliefs evolve over time with randomness and learning, revealing non-normal convergence behaviors.

Contribution

It develops a novel framework for studying agents' learning dynamics with stochastic shocks and provides a limit theorem for belief convergence in these models.

Findings

Beliefs converge in distribution, not necessarily normally.

New insights into stochastic social learning models.

Application of advanced techniques to random linear systems.

Abstract

We develop original models to study interacting agents in financial markets and in social networks. Within these models randomness is vital as a form of shock or news that decays with time. Agents learn from their observations and learning ability to interpret news or private information in time-varying networks. Under general assumption on the noise, a limit theorem is developed for the generalised DeGroot framework for certain type of conditions governing the learning. In this context, the agents beliefs (properly scaled) converge in distribution that is not necessarily normal. Fresh insights are gained not only from proposing a new setting for social learning models but also from using different techniques to study discrete time random linear dynamical systems.

Figures

Equations

$$x_{t}^{i}=\underbrace{f_{i,t}(X_{t-1})}_{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}averaging}+\underbrace{g_{i,t}(x_{t-1}^{i})}_{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}learning},$$

$$x_{t+1}^i=\sum _{j=1}^n(a_t{)}_{ij}{x}_t^j+{ϵ}_{i,t}(\sigma -x_t^i),$$

$$X_{t+1}=A_t{X}_t+{\mathcal{E}}_t(\sigma \text{mathbbm}{1}_n-X_t),$$

$${\rho }_t=\underset{i=1,\cdots ,n}{\max }(|(a_t{)}_{ii}-{ϵ}_{i,t}|+1-(a_t{)}_{ii}).$$

$$\underset{t\ge 1}{\sup }\{{\rho }_t+{\rho }_t{\rho }_{t-1}+{\rho }_t{\rho }_{t-1}{\rho }_{t-2}+\cdots +{\rho }_t{\rho }_{t-1}\dots {\rho }_1\}<\infty .$$

$$|v{|}_{\infty }=\underset{i=1,\cdots ,n}{\max }|v_i|\text{ and }|v{|}_1=\sum _{i=1}^n|v_i|$$

$$⟨v,w⟩=v^{\prime }w=\sum _{i=1}^n{v}_i{w}_i,$$

$$|v{|}_{\infty }=\underset{|w{|}_1\le 1}{\sup }⟨v,w⟩\text{ and }|v{|}_1=\underset{|w{|}_{\infty }\le 1}{\sup }⟨v,w⟩.$$

$$|B{|}_{\infty }=\underset{i=1,\dots ,m}{\sup }\sum _{j=1}^n|b_{ij}|\text{ and }|B{|}_1=\underset{j=1,\dots ,m}{\sup }\sum _{i=1}^n|b_{ij}|.$$

$$|Bv{|}_{\infty }\le |B{|}_{\infty }|v{|}_{\infty }.$$

$$|B{|}_{\infty }=\underset{|v{|}_{\infty }\le 1}{\sup }|Bv{|}_{\infty }.$$

$$X_t⟹X\text{ if }\mathbb{E}[\varphi (X_t)]\underset{t\to \infty }{\to }\mathbb{E}[X]$$

$$X_{t+1}=A{X}_t+\mathcal{E}(\bar{\sigma }-X_t).$$

$$\underset{t\to \infty }{\lim }{X}_t=\bar{\sigma }.$$

$$X_{t+1}=A{X}_t+\mathcal{E}(\bar{\sigma }-X_t).$$

$$X_{t+1}-\bar{\sigma }=(A-\mathcal{E})(X_t-\bar{\sigma }).$$

$$(Y_t{)}_i=\sum _{j=1}^n{b}_{ij}(Y_{t-1}{)}_j,$$

$$|(Y_t{)}_i|$$

$$\le |Y_{t-1}{|}_{\infty }\sum _{j=1}^n|b_{ij}|.$$

$$\sum _{j=1}^n|b_{ij}|=|a_{ii}-{ϵ}_i|+\sum _{j\ne i}^n|a_{ij}|=|a_{ii}-{ϵ}_i|+1-a_{ii},$$

$$\rho =\underset{\begin{array}{c}i\end{array}}{\max }(|a_{ii}-{ϵ}_i|+1-a_{ii}),$$

$$|Y_t{|}_{\infty }\le {\rho }^t|Y_0{|}_{\infty }.$$

$$|Y_t{|}_{\infty }\underset{t\to \infty }{\to }0$$

$${\rho }_t=\underset{i}{\max }(|(a_t{)}_{ii}-{ϵ}_i(t)|+1-(a_t{)}_{ii})$$

$$\prod _{i=1}^t{\rho }_i\underset{t\to \infty }{⟶}0,$$

$$|\nu {|}_{\infty ,\beta }=\underset{i=1,\cdots ,n}{\max }|{\nu }_i|/{\beta }_i$$

$$|(Y_t{)}_i|$$

$$\frac{|(Y_t{)}_i|}{{\beta }_i}$$

$$|(Y_t){|}_{\infty ,\beta }$$

$$=|(Y_{t-1}){|}_{\infty ,\beta }\underset{i=1,\cdots ,n}{\max }\left(|a_{ii}-{ϵ}_i|+\frac{1}{{\beta }_i}\sum _{j\ne i}^n{a}_{ij}{\beta }_j\right).$$

$$|Y_t{|}_{\infty ,\beta }\le |(Y_{t-1}){|}_{\infty ,\beta }\underset{i=1,\cdots ,n}{\max }\left(|a_{ii}-{ϵ}_i|+\delta -a_{ii}\right)$$

$$X_t=A_t{X}_{t-1}+{\mathcal{E}}_t(\bar{\sigma }+{\gamma }_t-X_{t-1})$$

Full text

1. 1 Introduction

2. 1.1 Basic models - Averaging + Learning

3. 1.2 General Noisy Models

4. 1.3 Limiting distributions

5. 1.4 Noisy Feedback and Related Literature

6. 1.5 Organization of the paper

7. 2 Notation and Assumptions

8. 3 Base Model

9. 4 Learning with random noise

10. 4.1 Noisy Learning

11. 4.2 Simulations for convergence to distribution

12. 5 Nonlinear learning

13. 6 Limit Theorems in Distribution for time invariant models

14. 6.1 Significance and interpretation of Theorem 13

15. 6.2 Simulation to Convergence and Numerical Discussion

16. 7 Conclusion

1 Introduction

How do agents reach consensus on prices in realistic noisy environments? This is the central theme of this paper. Traders interact with one another and learn from their own environment. While learning and interaction can be modelled satisfactorily, the introduction of noise complicates the choice of a framework and model. Furthermore, the choice of discrete or continuous time expands this issue. Our aim is to propose new models of interaction and learning where noise is a significant feature in discrete time. These new models of learning and interaction entail agents who observe actions of other traders and update their own beliefs. Repeated interaction can in certain cases lead to consensus on a particular value of a tradeable commodity. Interaction models should take into account the environment of trading. When agents interact in financial markets, they are visually, algorithmically observing each other other’s quotes.

Prices are updated fast and in a simultaneous manner but the environment is always noisy. Averaging models offer a cogent and natural way to analyse interaction when agents learn and observe each others’ past actions through an online platform. Visual screens provide a snapshot of where current prices are. For such models there is a rich interplay between probability, dynamical systems and game theoretic ideas [MT17]. In financial markets, trading is never sequential (one player at a time) nor is information perfectly perceived or received by agents. Transactions occurs at breakneck speed [Buc15]. Agents (algorithms) move simultaneously: cancellations are the norm in today’s fast markets. In practical terms, sequential learning models don’t seem appropriate [VMP20, ABMF21]. Interaction is important in the emergence of consensus. Choices by agents from the previous round of play are available to all agents in the current round of play. The question is then what sort of averaging or heuristic process is ideal?

DeGroot type of learning models convey an essential and robust idea that is taking a firmer foothold in theory [AAAGP21, ABMF21, MTSJ18]. They offer a functional form of updating. Myopic updating occurs in each round. Something akin to persuasion bias could explain our basic model [DVZ03]. As in an echo chamber, agents in our setup have fixed weights but update their responses until consensus is reached. One could think of it as a behavioural heuristic and why repeated averaging is effective. Alternatively, with the right cost function representing the distance of an agent’s opinion against other opinions, the best response is repeated averaging. Recently there have been some experimental papers on evidence of DeGroot updating [CLX20, BBC17]. Repeated averaging models are our base precisely because they capture the nature of interaction and learning in financial markets so succinctly. On top of the base models we develop more sophisticated extensions, relaxing the fixed nature of the weights and learning matrices.

Our discussion so far is on trading but any network where the players have access to some sort of learning feedback is suitable. A game theoretic framework where every player takes into account other players’ payoff is unrealistic and points to serious difficulties on how to even represent utilities [Kir02]; these are economic arguments that are better addressed by in depth philosophical interludes. Moreover, traders rarely have access to private information on how previous decisions led to a certain payoff for their opponent at least not in a high frequency sense. If a trading firm is a publicly listed company, then one can infer its trading losses or gains from public records. Nevertheless specific profit and loss accounts of trading individual stocks is a private matter. Firms never break down their income statements down to specific asset classes or instruments. Results are amalgamated and reported quarterly: not per hour, minute or second. Dynamical learning is an active area of research in computer science as well. Articles [PP18, PNGCS14, MPV17] propose and analyze the dynamics rather than just the concept of Nash equilibrium. Similar questions and issues to this paper were raised in [Kir02] at an informal level.

1.1 Basic models - Averaging + Learning

There are many models of noisy learning [AO11, BHOT05, LeB01, MPL17, MF13, Sob00, Sob20]. Depending on the question, different paradigms have been put forth. Our objective is learning and so we aim to use aspects of both social learning theory and dynamical systems. Difficulties in Bayesian environments mean the DeGroot model has become a workhorse for social learning [BBCM21, Sob00]. It offers a way forward for tractable models that can relax simple assumptions. Research using this framework is still active. In our setting, a group of traders observe quotes of others and incorporate an average of the previous quotes. The stark departure from standard DeGroot learning comes from the fact that not only are the agents averaging but they are getting imperfect feedback from an external/internal source on the true consensus value. To our knowledge, the setting of these types of consensus models to trading is new. We use the framework of [VMP18, VMP20] as the base case for our models, generalizing these models in the form

[TABLE]

where $X_{t}$ is the vector of previous beliefs of $n$ agents at time $t$ and $f_{i,t},g_{i,t}$ determine the dynamic for the $i$th agent. We treat the case where $f$ is linear and $g$ could be allowed to be non-linear. To keep things simple we will discuss in the introduction the full linear case given by

[TABLE]

which in the matrix form reads

[TABLE]

where $X_{t}=(x^{1}_{t},...,x^{n}_{t})^{T}$ is the opinion of each agent in discrete-time $t$, and $\mathcal{E}=\textrm{diag}(\epsilon_{1},...,\epsilon_{n})$ is the learning rate of each agent when they are provided with a known feedback $\sigma$. The stochastic matrix $A_{t}$ encapsulates the weights agents put on each other at time $t$ and is partially similar to the DeGroot model.

Agents’ aptitude to determine the quality of feedback is their ability $\epsilon_{i}$. A concrete example is in the options market. A trader observes quotes for volatility of an option in the market. At the same time, the option (with a particular strike and maturity) will have a theoretical known value of $\sigma$. The trader’s own ability to extract this theoretical model is conveyed by his or her institution’s quantitative model. Options, like other financial assets, have a replicable, theoretical value implied by a model. This theoretical value means that the feedback is interpreted as skill by the trader’s institution.

We should point out from the beginning that this model is not intended to be a DeGroot model. Our focus here is to study more conditions under which consensus arises. Thus, in this first round, the value of $\sigma$ is known to all players and the goal is to give conditions under which the interaction leads to convergence to the real value of $\sigma$. A key difference between our model and existing literature is that recursions of the form $X_{t}=A_{t}X_{t-1}+B_{t}$ in our setup have time dependent matrices. For systems with independent and identically distributed $A_{t},\,B_{t}$ there has been prior work [DSGLP04, DF99, BDG10, BDM*+*16, Kes73, PVFT22]. In traditional game-theoretic models the focus is on equilibrium [FDLL98]. The focus here is how agents behave in the long run, either they converge to a point or (properly scaled) to a distribution.

1.2 General Noisy Models

The base model is deterministic. The next natural and more realistic extension is to assume that the value of $\sigma$ is noisy. In this context, the model has a more dynamical interpretation, the agents do not observe the true value of $\sigma$, however our main focus is to study conditions under which the agents converge in some form.

We study this under various forms. The first type is the one in which the noise decreases with time to [math]. The other case is where the noise is persistent. We give some general conditions under which the convergence takes place. Under these general conditions on the interaction matrix and the learning rates, if the noise is decaying to [math] in some mathematical form, then the agents converge to $\sigma$. If the noise is permanent though, then, under some supplementary conditions, the convergence is in a distributional sense.

It is worth to mention, how we quantify this. The key point is to define

[TABLE]

Notice that this is easy to compute and depends only on the diagonal part of the matrix $A_{t}$ and the learning rates matrix $\mathcal{E}_{t}$. Our condition on convergence is driven by the following ensemble of these individual quantities as

[TABLE]

Notice that in the pure DeGroot model, $\rho_{t}=1$ for all $t$, the above condition is not satisfied which shows that the learning part plays an important role in the convergence.

Under another condition on the fluctuation of the matrices $A_{t}$ and $\mathcal{E}_{t}$, we show that if the noise in the feedback is iid, then the distribution of $X_{t}$ converges. However, the limiting distribution is not easy to describe. We point out that in the case the learning part $g$ in the model (1.1) is allowed to be non-linear, we can give a version of the quantity $\rho_{t}$ under which most of the results can be extended.

1.3 Limiting distributions

We prove that when the noise is iid, under some conditions, we have convergence in distribution of the agents’ opinions. However, the description of the limiting distribution is not explicit. To understand a little more the limiting distribution, we take the linear case where the matrices $A$ and $\mathcal{E}$ are time independent. Here we can prove that after rescaling the vector $X_{t}$, we can compute the asymptotic behavior. This asymptotic behavior is fully controlled by the Jordan decomposition of the matrix $A-\mathcal{E}$.

One of the possible behavior of the vector $X_{t}$ in the limit is governed by the normal distribution, very similar to the case of the Central Limit Theorem. Although, this is one of the several possibilities; the one in which some of the eigenvalues of $A-\mathcal{E}$ are on the unit circle and all the rest are inside the unit disk. In this case, the behavior is as one would expect, the noise appears in the limit through its covariance matrix and not its distribution.

What is really interesting to point out is that in addition to this behavior there are other behaviors. Indeed, if all the eigenvalues of $A-\epsilon$ are in the unit disk, then the vector $X_{t}$ converges without any additional scaling. However, the limit depends on the whole distribution of the noise.

In the remaining case, if the matrix $A-\mathcal{E}$ has eigenvalues of absolute value strictly bigger than 1, then $X_{t}$ grows exponentially and in addition could also oscillate. Overall, the whole distribution of the noise is involved in the limiting behavior of $X_{t}$.

For the general limiting behavior of $X_{t}$, we can observe in subsection 6.2 and Figure 2 distributions that could be fractal like depending on the noise. Important though is to mention that these limiting distributions could be used from the statistical standpoint to give estimates on the true parameter $\sigma$.

It is also worth emphasizing that the limit distribution of the scaled dynamic is extremely sensitive to the structure of the Jordan decomposition of the matrix $A-\mathcal{E}$ and in particular the spectrum of the matrix $A-\mathcal{E}$. Therefore, slight perturbation of the matrix could lead to radically different behavior of $X_{t}$ particularly in the case of eigenvalues of $A-\mathcal{E}$ in the proximity of the unit circle.

1.4 Noisy Feedback and Related Literature

The feedback can best explain the situation where a similar instrument is traded on another exchange or there is a common source of market chatter. Moreover, such chatter is commonly provided through voice box brokers or over-the-counter markets. We assume all agents have access to this feedback or chatter. Take for example trading in Foreign Exchange (FX). Trading in USD/Euro can occur in both London and New York, when both markets are open. During this period, traders from both markets can see what each group is quoting, and during the crossover period before the New York traders become dominant, quotes from both centres will be tightly linked. Their ability via $\varepsilon$ to capture feedback on this $\bar{\sigma}$ through either external markets or internal models is represented by $\epsilon_{i}(\sigma-x^{i}_{t})$. In reality, for G7 currencies, most of the spot trading will be done by algorithms tracking each other’s movements. At the same time these algorithms will have internal models guiding them to what the true theoretical value of the asset is, ensuring the price quoted is arbitrage free. Thus, feedback or learning is aptly represented and illustrated as a separate process to visible, external averaging. Another example would the S&P500 European ETF (SPY) options, which are not cash settled as SPX options but stock settled. Quotes for the SPY options will also be linked with the SPX options. There will always be a model or arbitrage-free implied theoretical value. Equities are traded across multiple exchanges and platforms like currencies. Sometimes trades occur off-exchange and get reported at the end of the day through the Exchange’s clearing system. This points to the fact that models with Averaging + Learning aspects are well suited for portraying trading dynamics as social phenomenon for a different types of financial assets. How agents interpret information or market chatter is their unique learning ability. This ability is impacted by noise. Learning is not perfect. No additional assumption is required apart from the fact that noise is either persistent or transient. In either case, the system settles down, converging to consensus or to a distribution. Noise can be either common to all players or distinct.

Related Literature

Previous literature has not addressed this noisy feedback in a sustained, systematic effort to investigate averaging models, partly because of the difficulties of discrete-time dynamics and asymptotic analysis. There have been some contributions focusing on empirical results [KB08, MB21]. While noise was added to a pure DeGroot model in [SL21], a Central Limit Theorem was not proven, rather noise added was assumed to be Gaussian; though this work did introduce global and local uniqueness noise models in its framework through simulations. Noise is also added in bounded confidence models [SWC*+*20] such as Hegselmann-Krause (HK) models, which are quite distinct from DeGroot dynamics [HK*+*02, SCH17, WLC*+*17]. Interestingly, but complementary to our work is that of $\frac{1}{m}$-DeGroot dynamics [AAAGP21]. A variant of DeGroot dynamics that is robust to stubborn agents and mis-specification is introduced. However, different from our framework, noise is present only at the beginning of the dynamics, after which the focus is on the interaction protocol with a large number of agents. This type of $\frac{1}{m}$-DeGroot dynamics approximates the standard DeGroot dynamics to the nearest fraction of the average of the players. Agents are restricted to choosing an opinion on the $\frac{1}{m}$ grid. It is a coarsening of the DeGroot dynamics. This model is in response to the vulnerability of stubborn agents leading consensus astray [ACFO13, GJ10, GS14]. In our framework, certainly agents can be stubborn and not interact. Thus the dynamics in 1.2 can feature a row with [math]’s and one $1$ for the agent not interacting or sticking to her previous opinion. Still, in our models the learning aspect through $\mathcal{\varepsilon}$ ensures every agent reaches asymptotic consensus. Unlike some Bayesian models, noise in our framework can be common or distinct to each agent [MST14, HJMR19]. Moreover, unlike Kinetic models of opinion exchange [DMPW09, DW15, Tos06], our models are in discrete-time so continuous-time techniques are not applicable. Time-varying networks have been studied in continuous-time frameworks in different settings [CF22, GFR*+*22, MPPD19]. However, our concept of noise in space and time is quite different from earlier works.

The DeGroot model arises as a special case in our framework, though our results do not apply to the pure DeGroot model. Noise in some Bayesian, social learning models is viewed as a private signal [Ban92, BBCM21, HJMR19]. However, the emphasis we put is on the probabilistic notions of consensus and realistic computations by agents as $t\to\infty$. This holds regardless of the number of agents. Our setting is general enough to incorporate a range of models in the feedback. For time varying models with just $A_{t}$, without noise, see [Bul22, LAJ08, Sha22, FF18, WTM19]. The recursions involving time-varying $A_{t}$ and $\mathcal{E}_{t}$, mean we are dealing with backward matrix products. However, the conditions we use for convergence are slightly weaker than even weak ergodicity [Bré13, CS77].

1.5 Organization of the paper

In section 2 we introduce the main definitions and notations. Then, in section 3 we discuss the base model, an extension of the result from [VMP18]. In section 4 we introduce the models with noise. Theorem 5 is the main result of this part. We treat the non-homogeneous and nonrandom model for the matrix $A_{t}$ and add external noise in $n$-dimensional Euclidean space.

We provide in section 5 a non-linear extension of the linear noisy model. Players still average from their observations of past actions but their own unique learning ability and how they interpret the extra information is a nonlinear function. This type of model fits with the earlier linear models, preserving the averaging nature of interaction. Suitable conditions on the nonlinear function are derived that exhibit consensus. Section 6 presents the convergence in distribution results in the case of time independent noisy model with iid noise. For more comments and relevance of the results see subsection 6.1.

2 Notation and Assumptions

In all subsequent analysis, $A$ refers to a row-stochastic weights matrix, whose rows sum to one. Depending on the setup, $A$ can be time varying or fixed.

We use the different norms, namely we take for a vector $v=\begin{bmatrix}v_{1}\\ \vdots\\ v_{n}\end{bmatrix},\,$

[TABLE]

and the inner product of two vectors $v,w$ is given by

[TABLE]

with the standard use of the transpose for $v^{\prime}=[v_{1},v_{2},\dots,v_{n}]$.

We have here a duality result computing one norm in terms of the inner product in the form

[TABLE]

For any $m\times n$ matrix $B$, we denote

[TABLE]

We then have for any $m\times n$ matrix $B$ and any $n$ dimensional vector $v$

[TABLE]

It is in fact easy to see that

[TABLE]

Finally, for a sequence of $n$-dimensional random variables $(X_{t})_{t\geq 0}$ and another $n$-dimensional random variable $X$ we use the notation

[TABLE]

for any continuous and bounded function $\phi:\mathbb{R}^{n}\to\mathbb{R}$.

3 Base Model

We further discuss our base models here to give the reader a perspective of the type of models analyzed. In the base model, we have $n$ agents and a fixed row-stochastic matrix $A$, which is the weights matrix. Consider the dynamics for updating

[TABLE]

We can impose a weaker condition on $\epsilon_{i}$ and use $\bar{\sigma}=\sigma\mathbbm{1}_{n}$ for notational convenience when the dimension is clear. The vector $\mathbbm{1}_{n}$ denotes the vector with all components equal to $1$.

Proposition 1**.**

If $0<\epsilon_{i}<2a_{ii}$, then all agents reach the same consensus value, in other words,

[TABLE]

Proof.

Recall the equation (3.1) of the dynamics

[TABLE]

Equation (3.1) can now be rewritten as

[TABLE]

Setting $B=(A-\mathcal{E})$ and $Y_{t}=X_{t+1}-\bar{\sigma}$, the updating rule simplifies to

[TABLE]

from which we can then obtain

[TABLE]

Therefore, $|Y_{t}|_{\infty}\leq|Y_{t-1}|_{\infty}\max_{\begin{subarray}{c}i=1,\cdots,n\end{subarray}}\sum_{j=1}^{n}|b_{ij}|$.

On the other hand $b_{ij}=a_{ij}$ if $i\neq j$ so that

[TABLE]

where we have used the stochasticity of $A$, that is, the sum of the elements of each row is 1. From this if we check that $|a_{ii}-\epsilon_{i}|+1-a_{ii}<1$ which is the same as $|a_{ii}-\epsilon_{i}|<a_{ii}$ or equivalently $0<\epsilon_{i}<2a_{ii}$, then with

[TABLE]

we definitely obtain $0\leq\rho<1$ and $|Y_{t}|_{\infty}\leq\rho\,|Y_{t-1}|_{\infty}$. This is enough to conclude that

[TABLE]

From which letting $t\to\infty$ shows that

[TABLE]

and in particular also proves that $Y_{t}\underset{t\to\infty}{\longrightarrow}0.$ ∎

We should point out that in the above result, the convergence to $\bar{\sigma}$ is exponential and in fact, from the proof, we have that $|X_{t}-\bar{\sigma}|_{\infty}\leq\rho^{t}|X_{0}-\bar{\sigma}|_{\infty}$. Thus $\rho$ is a rate of convergence, but it might not be the optimal one. The true rate of convergence might in fact be much smaller. This is dictated in principle by the spectral radius which in general is much smaller than the proposed quantity $\rho$ above. This is covered by Gershgorin’s theorem.

For instance, if A=\left[\begin{array}[]{rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr}1/2&1/2\\ 1/2&1/2\\ \end{array}\right], and we take $\epsilon_{1}=0.01$, while $\epsilon_{2}=0.99$, then the eigenvalues of $A-\mathcal{E}$ are $\lambda_{1}=-0.700071,\lambda_{2}=0.700071$ while $\rho=.99$. Furthermore, if we take $X_{0}$ to have equal components equal to $1/2$, then (X_{100}-\bar{\sigma})/\lambda_{2}^{100}=\left[\begin{array}[]{rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr}1/2\\ 1/3\\ \end{array}\right] showing that it converges to [math] much faster. The result here is a conservative one in the sense that the convergence is still exponential though we do not get the exact rate of convergence. This analysis works well if the matrix $A$ is time independent, but as soon as we allow $A$ to change with time, the eigenvalue and eigenvector analysis no longer applies.

For the case of constant matrix, one can have a much better understanding of the convergence rate by simply writing the matrix $A-\mathcal{E}$ in Jordan form as $A-\mathcal{E}=SJS^{-1}$, where $S$ is a matrix of eigenvalues and $D$ is a Jordan block matrix. From this, one can solve for $X_{t}=\bar{\sigma}+SJ^{t}S^{-1}(X_{0}-\bar{\sigma})$ and this gives a structure equation for $X_{t}$ with more details on the behavior of $X_{t}$ for large $t$. The decay to $\bar{\sigma}$ is clearly controlled by the eigenvalue with the largest absolute value and its coefficient is given by the corresponding eigenvector. In the case of eigenvalues with higher multiplicity, we have more contributions but still everything is in terms of the matrices $J$ and $S$.

The argument of Proposition 1 allows an extension to the case when the matrices $A_{t}$ and $\mathcal{E}_{t}$ depend on t. The bottom line here is that if we define

[TABLE]

and these quantities satisfy,

[TABLE]

then we still have convergence to $\bar{\sigma}$.

For example, this is the case if all $\rho_{t}$ are bounded by $\rho<1$. However, condition (3.2) also allows cases where $\rho_{t}\underset{t\to\infty}{\longrightarrow}1.$ We highlight two examples. For the first we have convergence.

Example 2**.**

Let’s consider $\rho_{t}=\frac{t}{t+1}$, then $\prod_{i=1}^{t}\rho_{i}=\frac{1}{t+1}$ which converges to 0 as $t\to\infty$.

However, condition (3.2) also ensures we don’t have the following situation.

Example 3**.**

Let’s consider $\rho_{t}=\exp(-\frac{1}{t^{2}})$, then $\prod_{i=1}^{t}\rho_{i}=\exp(-\sum_{k=1}^{t}\frac{1}{k^{2}})$ which does not converge to zero.

Condition (3.2) can also be written as $\sum_{i=1}^{t}\log\rho_{i}\xrightarrow[t\rightarrow\infty]{}-\infty,$ or differently as $\sum_{i=1}^{t}(-\log\rho_{i})\xrightarrow[t\rightarrow\infty]{}\infty$. In fact, this is the case if $\frac{-\log\rho_{t}}{t^{-\alpha}}\geq C$ for some $C>0$ and $\alpha>0$. This translates into $\rho_{t}\leq e^{-Ct^{\alpha}}$.

An astute reader would immediately notice that $\rho_{t}=|A|_{\infty}$ is one of the possible norms one can use. In fact we can replace the matrix norm with any matricial norm generated by a vector norm in $\mathbb{R}^{n}$. More precisely we can take $\rho_{t}=|A|_{\alpha}$ where $|A|_{\alpha}=\max_{|x|_{\alpha}=1}|Ax|_{\alpha}$ with $|\cdot|_{\alpha}$ chosen to be a vector norm on $\mathbb{R}^{n}$. For details on matricial norms, one can take a look at [HJ12]. However the infinity norm has a clean and clear interpretation in terms of $a_{ii}$ and $\epsilon_{i}$ and that is the main reason we use it throughout the paper.

We can extend the conclusions if we replace the $\infty$-norm of a vector by something of the form

[TABLE]

where $\beta$ is a vector of positive values such that $A\beta\leq\delta\beta$. In this new norm we now have

[TABLE]

which yields

[TABLE]

From the assumption $A\beta\leq\delta\beta$ we can get in the first place that $\sum_{j=1}^{n}a_{ij}\beta_{j}\leq\delta\beta_{i}$ or $\sum_{j\neq i}^{n}a_{ij}\beta_{j}\leq\beta_{i}(\delta-a_{ii})$ and thus $\frac{1}{\beta_{i}}\sum_{j\neq i}^{n}a_{ij}\beta_{j}\leq(\delta-a_{ii})$. This yields

[TABLE]

as long as $|a_{ii}-\epsilon_{i}|+\delta-a_{ii}<1$, which is satisfied by $-(1-\delta)<\epsilon_{i}<1-\delta+2a_{ii}$. The question is if there exists such a vector with $A\beta\leq\delta\beta$ (this means component wise). Such a choice is $\beta=[1,1,\dots,1]^{\prime}$ and $\delta=1$ since $A$ is a stochastic matrix. If such a $\beta$ exists with $\delta<1$ then we get a relaxation of the main condition.

Interestingly, if $A$ is not necessarily stochastic but has positive entries, then by a theorem of Perron-Frobenius there exists a real eigenvalue that is greater than the absolute value of all the other eigenvalues and its eigenvector has positive entries. The argument above shows that we can definitely choose $\delta$ and $\beta$ to have the same result.

The above arguments allow us to posit this result.

Theorem 4**.**

Assume $X_{t}=A_{t}X_{t-1}+\mathcal{E}_{t}(\bar{\sigma}-X_{t-1})$ with $A_{t}$ row-stochastic matrix and let $\rho_{t}=\max_{i=1,\cdots,n}(|(a_{t})_{ii}-(\epsilon_{t})_{i}|+1-(a_{t})_{ii})$. If   $\prod_{s=1}^{t}\rho_{s}\underset{t\to\infty}{\longrightarrow}0$, then $X_{t}\underset{t\to\infty}{\longrightarrow}\bar{\sigma}$.

In the case $A_{t}$ are all equal to A, then if $0<\epsilon_{i}<2a_{ii},\>i=1,\cdots,n,$ then $X_{t}\underset{t\to\infty}{\longrightarrow}\bar{\sigma}$.

4 Learning with random noise

Our base model with learning is expanded to have random noise in the feedback term. We introduce a random vector $\gamma_{t}$ which we quantify later. The hypothesis is that $\gamma_{t}$ is small. For this section we also consider the case of time depending evolution.

The model is given by

[TABLE]

where $X_{t}$ is the vector of prices at time $t$ and $\bar{\sigma}$ is the vector of equilibrium price or consensus value the agents are trying to learn. In order to prove that $X_{t}-\bar{\sigma}$ converges to 0, we rewrite the equation as

[TABLE]

Therefore if we denote by $Y_{t}=X_{t-1}-\bar{\sigma}$, then we can simplify the above expression as

[TABLE]

With the same argument as before we obtain

[TABLE]

with

[TABLE]

We formulate a general result as follows.

4.1 Noisy Learning

In the theorem below, we examine the appropriate noise in convergence terms. With vanishing noise, the system still exhibits the consensus property. Proving this convergence with vanishing noise in probability requires a separate lemma. Noise has two parts in this theorem. In the first three parts, noise is vanishing and consensus is reached. In markets, this noise is seen as a form of shock that decays over time. The Theorem quantifies this precisely, using distinct modes of convergence. In the fourth item, when the noise is persistent agents do not reach consensus and also do not converge to the same asymptotic distributions. So while $X_{t}$ converges asymptotically, the individual agents may converge to different marginal distributions.

Theorem 5**.**

Assume the model $X_{t}=A_{t}X_{t-1}+\mathcal{E}_{t}(\bar{\sigma}+\gamma_{t}-X_{t-1})$ with $A_{t}$ a row-stochastic matrix. With the notation from (4.1) assume that

[TABLE]

If $\gamma_{t}\xrightarrow[t\to\infty]{a.s}0$, then $X_{t}\xrightarrow[t\to\infty]{a.s}\bar{\sigma}$. 2. 2.

If $\gamma_{t}\xrightarrow[t\to\infty]{\mathbb{P}}0$, then $X_{t}\xrightarrow[t\to\infty]{\mathbb{P}}\bar{\sigma}$. 3. 3.

If $\gamma_{t}\xrightarrow[t\to\infty]{L^{p}}0$, then $X_{t}\xrightarrow[t\to\infty]{L^{p}}\bar{\sigma}$. 4. 4.

If we assume

[TABLE]

where now $(\gamma_{t})_{t\geq 1}$ are iid and integrable and in addition to (4.2) we assume that

[TABLE]

Then, there exists an $n$-dimensional random variable $X$ such that

[TABLE]

where the convergence is distribution as pointed in (2.2). 5. 5.

Furthermore, if $\gamma_{t}$ is integrable but not constant almost surely, then, without condition (4.4), the conclusion of (4.5) does not hold.

Proof.

From our base model in terms of $Y_{t}$ is

[TABLE]

From this we get

[TABLE]

If we assume that $|\gamma_{t}|_{\infty}\xrightarrow[t\to\infty]{a.s}0$, then we get that $|Y_{t}|\xrightarrow[t\to\infty]{a.s}0$. Indeed, this becomes a purely deterministic statement. For a given $\epsilon>0$, we can find that $|\gamma_{t}|_{\infty}\leq\epsilon$ for all $t\geq t_{\epsilon}$. Then,

[TABLE]

Using the previous inequalities for $t-1$, $t-2$, …, $t_{\epsilon}$ gives that

[TABLE]

From (4.2) combined with the following elementary lemma we show that $|Y_{t}|_{\infty}\xrightarrow[t\to\infty]{a.s.}0$.

Lemma 6**.**

Assume that $\{\rho_{t}\}_{t\geq 1}$ is a sequence of non-negative numbers such that for some $A>0$, and any $t\geq 1$,

[TABLE]

Then, for $0\leq s\leq t-1$,

[TABLE]

and in addition,

[TABLE]

It is also true that (4.9) for some constants $c>0$ and $A>0$ implies (4.8) with the bound on the right being $A/(e^{c}-1)$.

Proof.

To see this we first denote

[TABLE]

Then we get that

[TABLE]

and thus

[TABLE]

To see (4.10), we only need to notice that

[TABLE]

It is a simple exercise to go from (4.9) back to (4.8). ∎ 2. 2.

If we only assume a weaker condition, namely that $\gamma_{t}\xrightarrow[t\to\infty]{\mathbb{P}}0$ (only convergence in probability), then iterating (4.7) we obtain

[TABLE]

with the convention that $\prod_{i=t+1}^{t}\rho_{i}=1$.

To finish the proof off we use the following Lemma with $u_{t}=|\gamma_{t}|_{\infty}$.

Lemma 7**.**

Let $(u_{n})_{n\geq 1}$ be a random sequence such that

[TABLE]

Then, under the assumption (4.2), we have the convergence $\sum_{i=1}^{t}\rho_{t}\rho_{t-1}\dots\rho_{t-i+1}u_{t-i}\xrightarrow[t\to\infty]{\mathbb{P}}0.$

Proof.

For the argument, denote for simplicity of writing $\eta_{t,i}=\rho_{t}\rho_{t-i}\dots\rho_{t-i+1}$.

Now, we fix $s\leq t$ and write

[TABLE]

Now, for a given $\epsilon$ and $|\sum_{i=1}^{t}\eta_{t,i}u_{t-i}|>\epsilon$, we must have that at least one of the above sums must be at least $\epsilon/2$, thus, we can write for each fixed $\epsilon>0$,

[TABLE]

The next step is to use the boundedness of $u_{t}$. Take arbitrary $\delta,M>0$, (here $\delta$ is meant to be small and $M$ to be large) and then set

[TABLE]

From the condition (4.13) we definitely have that $\mathbb{P}(A_{M})$ converges to $1$ as $M$ tends to infinity. Therefore we can continue the equation (4.14) with

[TABLE]

where in the passage from the first line to the second we used the union bound, more precisely, if we have $\sum_{i=1}^{s}\eta_{t,i}|u_{t-i}|\geq\epsilon/2$ then at least one of the terms must be $\geq\epsilon/(2s)$ plus the union bound on the probability. Finally in passage to the last line we simply used (4.10).

Next we can freeze for now $\epsilon,s,M$ and use the fact that for each $i$, $\eta_{t,i}u_{t-i}$ converges to [math] in probability since $\eta_{t,i}$ is bounded by $A>0$ and use (4.10) to argue that the limit as $t\to\infty$ we gain that

[TABLE]

For large $s$, obviously $\mathbb{P}(A^{2}e^{-cs}>\epsilon/(2M))=0$ and thus we arrive at

[TABLE]

From this, we take the limit as $M\to\infty$ and using (4.12)

[TABLE]

which means convergence of $\sum_{i=1}^{t}\eta_{i}u_{t-i}$ to [math] in probability. ∎

Now let’s return to the proof of the Theorem. 3. 3.

For the $L^{p}$ convergence we just need to take expectation of (4.11). 4. 4.

For the convergence in distribution we start by writing

[TABLE]

where $B_{t}=A_{t}-\mathcal{E}_{t}$. The idea is that because $\gamma_{t}$ are in $L^{1}$ so are all the variables $X_{t}$. We are going to use the Wasserstein distance to control the difference between the distributions of $X_{t}$ and $X_{t-1}$.

The basic idea is that in a slightly modified Wasserstein distance $D$ we have a contraction in the sense that there exists some $\rho<1$ such that

[TABLE]

For the sake of completeness we define here for two $n$-dimensional random variables, $X,Y$ or better for their distributions $\mu_{X},\mu_{Y}$,

[TABLE]

where $\alpha$ is a $2n$-dimensional distribution with marginals $\mu_{X}$ and $\mu_{Y}$ and $\tilde{X}$ $\tilde{Y}$ are two random variables on the same probability space (we call it a coupling) with the same distributions as $X$, respectively $Y$. The second equality follows easily from taking $\tilde{X}$ and $\tilde{Y}$ to be the projections from $\pi_{i}:\mathbb{R}^{n}\times\mathbb{R}^{n}\to\mathbb{R}^{n}$, given by $\pi_{1}(x,y)=x$ while $\pi_{2}(x,y)=y$. To go from the pair $(\tilde{X},\tilde{Y})$ back to the measure $\alpha$, we just need to take $\alpha$ to be the distribution of the pair $(\tilde{X},\tilde{Y})$.

The standard Wasserstein distance is defined as

[TABLE]

Because any two norms on $\mathbb{R}^{n}$ are equivalent, we can find two constants $c_{1},c_{2}>0$ such that

[TABLE]

It is known that $W_{1}$ gives the topology of weak converge on the space of probability measures with finite first moment (that is $\int|x|\mu(dx)<\infty$). Due to the above inequality we also infer the completeness with respect to the metric $D$ on the same space $\mathcal{P}_{1}(\mathbb{R}^{n})$.

To carry on this program we define for a distribution $\mu$, the following map

[TABLE]

where $X$ is a random variable with distribution $\mu$ and $\gamma$ is a random variable independent of $X$ and having the same distribution as the sequence $\gamma_{t}$.

Now we want to look at $D(X_{t},X_{t-1})$ and estimate it from above. To do this assume that we have a coupling between $X_{t-1}$ and $X_{t-2}$ and then we can create an optimal coupling between $X_{t}$ and $X_{t-1}$ (with respect to the distance $D$, which certainly exists from Kantorovich general result) and then take $\gamma$ independent of both $X_{t-1}$ and $X_{t-2}$ and use

[TABLE]

Taking $|\cdot|_{\infty}$ and the expectation both sides we get the estimate

[TABLE]

where we denoted by

[TABLE]

Notice that in the time independent case, the terms $\alpha_{t}$ is 0, which implies that $X_{t}$ converges in distribution.

In the general case we need to use the extra conditions from (4.4). From the above considerations we actually show first that the expectation of $X_{t}$ obeys the equation (keep in mind that $\sup_{t\geq 1}|\mathcal{E}_{t}|_{\infty}\leq A+1$)

[TABLE]

Using this and the standard iterations combined with (4.2) we get that

[TABLE]

On the other hand from (4.6) we get that

[TABLE]

Using this and a simple iteration it leads to

[TABLE]

In particular, summing this over $t$ from $t$ to $t+s$, leads to

[TABLE]

According to (4.10) we conclude that the sum $\sum_{i=1}^{s}\rho_{t+i-1}\dots\rho_{2}$ converges to [math] as $s,t\to\infty$. We will show that the other sum also converges to [math] as both $t,s\to\infty$. To this end notice that from (4.4), we can set

[TABLE]

and write $\alpha_{t}=\beta_{t}-\beta_{t+1}$. After rearrangements, this leads to

[TABLE]

The first term converges to 0 because of (4.9) and the rest, converges to [math] because of Lemma 7 thanks to the fact that $\beta_{t}$ converges to [math], this converges to [math].

This proves the convergence in distribution. 5. 5.

Next we show that the condition (4.4) is also a necessary condition. Indeed, if we take the one dimensional case with

[TABLE]

such that

[TABLE]

In fact we will choose

[TABLE]

and we will choose $w_{i}=\pm 1$ in the following fashion. First we take all $w_{1},w_{2},\dots,w_{\tau_{1}}$ such that $\epsilon_{\tau_{1}}\leq 3/4$ but $3/4<\epsilon_{\tau_{1}}+c/(\tau_{1}+1)$. Notice that we can do this because the harmonic series is divergent. Now, we choose $\tau_{2}>\tau_{1}$ such that $w_{\tau_{1}+1}=w_{\tau_{1}+2}=\dots=w_{\tau_{2}}=-1$ and $\epsilon_{\tau_{2}}-1/(10(\tau_{2}+1))<1/4\leq\epsilon_{\tau_{2}}$. Now we choose $\tau_{3}>\tau_{2}$ and $w_{t_{2}+1}=\dots=w_{t_{3}}=1$ such that $\epsilon_{\tau_{3}}\leq 3/4<\epsilon_{\tau_{3}}+c/(\tau_{3}+1)$. Then we choose $\tau_{4}>\tau_{3}$ such that $w_{\tau_{3}+1}=w_{\tau_{3}+2}=\dots=w_{\tau_{4}}=-1$ such that $\epsilon_{\tau_{4}}-1/(10(\tau_{4}+1))<1/4\leq\epsilon_{\tau_{4}}$. And we continue inductively. Thus we have defined a sequence $\epsilon_{t}$ such that

[TABLE]

In other words the limit points of the sequence $\epsilon_{t}$ is just the interval $[1/4,3/4]$ and obviously the condition (4.2) is fulfilled.

With this choice of the sequence $\epsilon_{t}$, we claim that the sequence $X_{t}$ does not converge in distribution. Indeed the argument is based on the simple observation that if it were, then taking the characteristic functions $\phi_{X_{t}}$ we would get

[TABLE]

As a recall, $\phi_{X}(\xi)=\mathbb{E}[e^{i\xi X}]$ for any $\xi\in\mathbb{R}$. In particular this means that if $X_{t}$ converges to some random variable $Y$, then taking a subsequence $t_{n}$ for which $\epsilon_{t_{n}}\xrightarrow[n\to\infty]{}x$ we obtain that

[TABLE]

Under the assumption that $\gamma$ is integrable we claim that $\gamma$ must be constant and also $X$ is going to be the same constant. To carry this out we argue that for $x=1/4$ and $x=3/4$ we get that

[TABLE]

Replacing $\xi$ by $4\xi/3$ we arrive at

[TABLE]

Replacing here $\xi$ by $\xi/3$, $\xi/3^{2}$, …, $\xi/3^{n}$ and multiplying these we obtain

[TABLE]

Now letting $n\to\infty$ and using the fact that for any random variable $Z$, $\phi_{Z}(\xi/3^{n})\xrightarrow[n\to\infty]{}1$ we obtain that

[TABLE]

in other words, $Y$ has the same distribution as $\gamma$. Using this in (4.19) with $x=1/2$ we arrive at

[TABLE]

Iterating this we get

[TABLE]

which can be written alternatively as

[TABLE]

where $Y_{1},Y_{2},\dots$ are iid with the same distribution as $Y$. Since $Y$ and $\gamma$ have the same distributions and $\gamma$ is integrable, it follows that $Y$ is also integrable. This in particular implies from the law of large numbers that $\frac{Y_{1}+Y_{2}+\dots+Y_{2^{n}}}{2^{n}}$ converges almost surely to $\mathbb{E}[Y]=\mathbb{E}[\gamma]$. Since convergence almost surely implies convergence in distribution, we get that

[TABLE]

in other words, $Y$ must be constant. This implies that $\gamma$ is also constant which then finishes the argument.

∎

As the proof is quite lengthy and requires two technical lemmas, we defer it to the appendix. Observe here the fact that in the last part of the Theorem we incorporated the constant $\bar{\sigma}$ into $\gamma_{t}$. The convergence is in distribution sense and thus it does not lead to convergence as in the previous cases. Even if we assume that $\gamma_{t}$ is of the form $\bar{\sigma}+\gamma_{t}$, the convergence will not be to $\bar{\sigma}$ alone. Thus this is a different convergence scenario and in spirit is not of the same form as the other cases.

Remark 8**.**

We need to point out that integrability is key for the conclusion of the last part of Theorem 5. This shows the intricate relationship (4.4) and (4.19). If we drop the integrability and for instance take $(\gamma_{t})_{t\geq 1}$ to be all iid Cauchy($1$) and $X_{0}=0$, then $X_{t}$ will also follow a Cauchy($1$) random variable for any choice of $0\leq\epsilon_{t}\leq 1$ with $\epsilon_{1}>0$. Certainly, in this situation we do not need any other assumptions on $\epsilon$ or $\rho_{t}$ or condition (4.4) to get convergence of $X_{t}$. We leave as an open problem the optimal conditions under which the model (4.3) converges as $t\to\infty$.

Condition (4.4) elucidates a key behavioural aspect. Agents are comfortable and so stabilize their trust matrix $A$ and learning ability $\mathcal{E}$. Like members of a small village or an island, everyone knows over time how much trust to place on themselves and the other members.

4.2 Simulations for convergence to distribution

Let us illustrate Theorem 5 and result 4.5. Suppose that the noise $\gamma_{t}$ is a Normal random variable. Numerical simulations show that $X_{t}$ converges to a Gaussian random variable for each component: figure 1.

The the asymptotic distribution is Gaussian centered around the true value $\bar{\sigma}$. The key point is that we do not need to scale $X_{t}$. Suppose, the iid $(\gamma_{t})s$ are vectors of just $+1$ or $-1$, then $X_{t}$ converges in distribution. In figure 1 (b), the simulated distribution looks distinctly non-Gaussian. For other noises, different distributions can occur. Note that even if agents converge in distribution, their marginal distributions can be distinct.

5 Nonlinear learning

While Averaging (updating) is retained in this section, we develop nonlinear models of learning. Instead of $\mathcal{E}$, there is a non-linear function.

Definition 9**.**

The learning function is $f_{t}:\mathbb{R}^{n}\to\mathbb{R}^{n}$ is continuous on some compact convex subset $K\subseteq\mathbb{R}^{n}$ and differentiable on its interior, with $f(0)=0$. Component wise it is

[TABLE]

Notice that the update or feedback is now varying with time. Learning or feed back stops when $\bar{\sigma}-X_{t}=0$, so the condition $f_{t}(0)=0$ ensures this. The updating rule for agent $i$ becomes

[TABLE]

Moreover, the weights matrix $A_{t}$ is also time varying and row-stochastic. Previous sections showed convergence results of linear updating $f_{t,i}=\epsilon_{i}$, a fixed scalar. Actual updating of feedback can be be quite complex, and having a nonlinear feedback or learning rule allows us to expand the linear model.

Theorem 10**.**

For $\forall i\in\{1,\cdots,n\}\mbox{ and }\forall t\geq 0$, suppose the learning function satisfies

[TABLE]

and if we denote

[TABLE]

and we assume that

[TABLE]

Then,

With the dynamics

[TABLE]

consensus is reached and $\lim_{t\to\infty}X_{t}=\bar{\sigma}$. 2. 2.

If the evolution is given by

[TABLE]

under the same assumption as in (5.1), then $\gamma_{t}\xrightarrow[t\to\infty]{}0$ yields that $X_{t}\xrightarrow[t\to\infty]{}\bar{\sigma}$. (If the noise converges to zero $a.s$, in probability or in ${L^{1}}$, then $X_{t}$ converges accordingly). 3. 3.

Again assume (5.2) and

[TABLE]

where the sequence $(\gamma_{t})_{t\geq 1}$ is assumed to be iid and integrable. If in addition we have that

[TABLE]

then $X_{t}$ converges in distribution as $t\to\infty$ as in the definition of equation (2.2).

Proof.

First we subtract $\bar{\sigma}$ from both sides of the dynamics equation. As $A$ is stochastic, $A(t)\bar{\sigma}=\bar{\sigma}$, hence

[TABLE]

Second, we recast the equation using the infinity-norm

[TABLE]

For individual $i$, the updating rule becomes

[TABLE]

The second equality follows because the learning function is continuous and differentiable hence

[TABLE]

for some $\xi_{i}\in(0,x)$ by the Mean value theorem.

By assumption

[TABLE]

but this is equivalent there being some $0<\delta_{i}<1$ such that $\forall\xi\in\mathbb{R}$

[TABLE]

The above condition gives us two cases to consider. In the first case, ignoring dependence on t, for all $i\in\{1,\cdots,n\}$ and $\xi$

[TABLE]

In the second case,

[TABLE]

Thus we obtain that

[TABLE]

thus we have a contraction in $|X_{t}-\bar{\sigma}|_{\infty}$ and consequently,

[TABLE] 2. 2.

The deviation equation from consensus is

[TABLE]

Essentially the same steps follow as the in the proof with no noise

[TABLE]

The rest of the proof follows as in the proof of Theorem 5, more precisely, following the same argument starting with (4.7). In all instances the convergence follows the same arguments as in the linear case. 3. 3.

First observe that from (5.3) we get

[TABLE]

From this, iterating and using (5.2) as in the linear case we obtain that

[TABLE]

To treat the case where $\gamma_{t}$ are all iid, we follow the same argument as the linear case. Here we have to use in the first place the distance defined in (4.16) and the argument for the estimate of $D(X_{t},X_{t-1})$ we need to take a for any coupling $\tilde{X}_{t-1}$ and $\tilde{X}_{t-2}$ the coupling $A_{t}\tilde{X}_{t-1}+f_{t}(\gamma-\tilde{X}_{t-1})$ and $A_{t-1}\tilde{X}_{t-2}+f_{t-1}(\gamma-\tilde{X}_{t-2})$. Then,

[TABLE]

From this we proceed exactly in the same way as in the proof of the linear case, more precisely, the same proof following (4.18) to show that $X_{t}$ is Cauchy in the metric $D$.

∎

Notice that the last part of the result above does not involve $\bar{\sigma}$ because it is actually hidden in the sequence $\gamma$. As opposed to the other two cases, the convergence is only in distribution and in principle that is implicitly defined, it is not a constant variable as in the previous cases.

Remark 11**.**

Matrix $A(t)$ and learning function $f_{t}$ are allowed to be time dependent or slowly varying. They could be random but in a controlled way. Were $A$ and $f$ to be fixed in time, the above result would still hold. So the constant case is a special case of what we have shown.

Continuity of the learning function $f_{t}$ is essential. We give an example of a situation where it breaks down.

Example 12**.**

Consider the sign function $\operatorname{sign}(x)=1$ for $x>0$, $\operatorname{sign}(0)=0$ and $\operatorname{sign}(x)=-1$ for $x<-1$. If the learning $f$ were the sign function, then the dynamics would be

[TABLE]

Consensus in this circumstance would not be achieved. One can plainly see this in the one dimensional case of $A_{t}=1$, $\sigma=1$, $Y_{t}=X_{t}-\sigma$, $Y_{0}=1$ and take $1/3<\mathcal{E}<1/2$. With this setup we get

[TABLE]

which shows that $Y_{t}$ becomes periodic, thus not convergent. We can extend this behavior to more general situations of course, though this periodic pattern still follows.

6 Limit Theorems in Distribution for time invariant models

We study a CLT result in the case of constant $A$ and $\mathcal{E}$. The analysis for Theorem 5 required condition (4.2). However, the use of this condition in the dynamics or iterations means a CLT is not possible. To obtain a general CLT result, we have to change this condition. Moreover, we examine the dynamics in more general form. In the previous theorem, to ensure consensus, each agent had to interact and learn. While there may be periods of no learning $(\epsilon_{t})_{ii}=0$ for some players in which case $\rho_{t}\geq 1$ for short bouts, eventually all agents have to learn and have positive self-belief $(a_{t})_{ii}$. To see this from a much higher perspective, we consider here the case of constant Average plus Learning matrices and study the limiting behavior of the beliefs in which the noise is added to the model. Thus we have

[TABLE]

where the noise $\gamma$ is assumed iid. In fact, we can continue with the model we studied above where the noise was inside the learning part, namely

[TABLE]

However, within the assumption that $A$ and $\mathcal{E}$ are constant we can simply redefine $\tilde{\gamma}_{t}=\mathcal{E}\gamma_{t}$ and with this change the above equation becomes

[TABLE]

which is essentially the model (6.1). We take one more step and rewrite the equations (6.1) in the form

[TABLE]

Assume that the matrix $A-\mathcal{E}$ has a standard Jordan form

[TABLE]

where $J$ is the Jordan decomposition of $A-\mathcal{E}$ with the blocks $(J_{k})_{k=1,\dots,l}$ on the diagonal and $J_{k}$ having dimension $m_{k}\times m_{k}$ and being defined by the eigenvalue $\lambda_{k}$. Here we can take the complex Jordan decomposition or the real decomposition. The computations are cleaner with the complex decomposition however the statements we are going to make are easily transferable to the real case as well.

Now consider

[TABLE]

and set $\mathcal{W}=\cup_{i\in W}\{\sum_{j=1}^{i}m_{j},\dots,(\sum_{j=1}^{j+1}m_{j})-1\}$ and similarly

[TABLE]

which represents the index set in $\{1,2,\dots,n\}$ corresponding to the Jordan blocks $J_{i}$ with $i$ in $W$ or $W_{max}$. Denote by

[TABLE]

where $J_{W}$ ($J_{W_{max}}$) is the block matrix where only the blocks with indices contained in $W$ (or $W_{max}$) appear, all the others having been replaced by 0.

Furthermore, we also introduce the matrices

[TABLE]

as the diagonal of the matrices $J_{W}$, $J_{W_{max}}$ respectively as the off diagonal parts of $J_{W}$, $J_{W_{max}}$.

In addition to these we will consider the following matrices

[TABLE]

where the inverse $D_{W}^{\#}$ is defined as matrix with the inverses of the non-zero blocks. In general $D_{W}^{\#}$ is not invertible and the non-zero elements on the diagonal of $\alpha D^{\#}$ are on the unit circle. Similarly, we define

[TABLE]

As an example, assume that the Jordan block matrix $J$ is the following

[TABLE]

where $\lambda_{1},\lambda_{2},\lambda_{3}$ are complex numbers. Clearly there are three Jordan blocks.

In this case, if $|\lambda_{1}|>\max\{|\lambda_{2}|,|\lambda_{3}|\}$, then $\alpha=|\lambda_{1}|$ and the quantities described above become $W=\{1\},m=2,W_{max}=\{1\}$ and

[TABLE]

with

[TABLE]

In the case we have $|\lambda_{1}|=|\lambda_{2}|>|\lambda_{3}|$, then we obtain $W=\{1,2\},m=2,W_{max}=\{1,2\}$ and

[TABLE]

with

[TABLE]

In the case we have $|\lambda_{3}|>\max\{|\lambda_{1}|,|\lambda_{2}|\}$, then we obtain $W=\{3\},m=1,W_{max}=\{3\}$ and

[TABLE]

with

[TABLE]

Now we can state the main result of this section. In Case I, when $\alpha<1$, we have convergence to a distribution. There is no scaling. In the original model of learning with non-varying averaging matrix and learning rates, in the case, $\mathcal{E}>0$, the iterations have a contraction and thus this result is similar in spirit to the case of the time varying case with persistent noise and the fourth item of Theorem 5. However we provide here an explicit description of the limiting distribution. In the theorem below we consider the other two cases of $\alpha>1$ and $\alpha=1$. It should be pointed out that Case III covers a pure DeGroot model with no learning: $\mathcal{E}=0$. For Case II, the interpretation could be that the players are putting too much weight on their feedback or learning ability or the $L^{1}$ norm of $A-\mathcal{E}$ is greater than 1. Thus, Theorem 5 and Theorem 13 complement each other by addressing different features of Averaging plus Learning.

Theorem 13**.**

Assume that $X_{t}$ satisfies

[TABLE]

where $(\gamma_{t})_{t\geq 0}$ is an iid sequence of random variables. Using the decomposition (6.3), we denote

[TABLE]

where $l$ is the number of Jordan blocks and $\lambda_{i}$ is the eigenvalue of each Jordan block. Then,

1. Case I.

If $\alpha<1$ and the noise $\gamma_{t}$ is in $L^{1}$, then

[TABLE] 2. Case II.

If $\alpha>1$ and the noise $\gamma_{t}$ is in $L^{1}$, then we can write

[TABLE]

where $R_{t}$ converges to [math] in $L^{1}$. 3. Case III.

If $\alpha=1$, and the noise $\gamma_{t}$ is $L^{2}$, iid with mean $\mu$ and covariance matrix $\Gamma$, then

[TABLE]

where the convergence is in weak/distribution sense and the covariance matrix $C$ is given by

[TABLE]

with the important convention that for $m=1$, $K^{0}=Id$ for any matrix $K$ (including the zero matrix).

Proof.

Before we start the proof we point out that the key to the analysis here is the Jordan decomposition. We will use here the convention that the Jordan blocks are real valued, however we use the complex version for the sake of the exposition. The real case can be worked out in a similar fashion with a little bit more care of the algebra. For a clarification, we point out that the real Jordan decomposition can be realized from the complex decomposition.

Using the decomposition $A-\mathcal{E}=P^{-1}JP$ and denoting $\tilde{X}_{t}=PX_{t}$, then we get

[TABLE]

Because the matrix $J$ is a block diagonal matrix, we can reduce the analysis to each block. The general result then follows by transferring the results to $X_{t}=P^{-1}\tilde{X}_{t}$.

We will treat each case separately for each Jordan block. In this case we fix an index $k$ and write $J=J_{k}$ as

[TABLE]

where $\lambda=\lambda_{k}$ and

[TABLE]

To simply the exposition here we assume that $m=m_{k}$ is the dimension of the Jordan block.

Notice the important fact that $L^{m}=0$. The key property which follows from this is that

[TABLE]

with the convention that $a\wedge b=\min\{a,b\}$.

Now we consider the first case of our result, namely $\alpha<1$. In this case, we denote by $Y_{t}$ the coordinates of $\tilde{X}_{t}$ corresponding to the Jordan block $J$ and let also $\delta_{t}$ the same corresponding part of $P\gamma_{t}$. Then, an easy algebraic calculation gives (cf. (6.12)) that

[TABLE]

Because $|\lambda|<1$, it is not difficult to observe (essentially from (6.12)) that $J^{t}$ converges to [math] as $t$ converges to $\infty$. In particular what this means is that $J^{t}Y_{0}$ converges to [math] as $t$ converges to infinity. On the other hand, the second term, namely $\sum_{s=0}^{t}J^{s}\delta_{t-s}$ has the same distribution as $\sum_{s=0}^{t}J^{s}\delta_{s}$. This sum is convergent in $L^{1}$. Indeed if $\lambda=0$, then $J^{t}=0$ for $t\geq m$. On the other case if $0<|\lambda|<1$ then, for example using (6.12), we can assure that $\alpha^{-t}J^{s}$ is a bounded matrix, thus in particular we obtain that $|J^{s}|\leq C\alpha^{s}$ for all $s\geq 1$, consequently it is now an elementary task to show that the series $\sum_{s=0}^{\infty}J^{s}\delta_{s}$ is convergent in $L^{1}$. Putting all the pieces together, we can easily see the conclusion.

For the second case, $\alpha>1$, we use again a reduction to blocks analysis. For a given block, we write now as above

[TABLE]

As opposed to the previous case when $|\lambda|<1$ we now look at the

[TABLE]

From the above expression, there are two terms we need to take into account. Now, using (6.12) for a given $s=0,1,2,\dots$ we analyze the asymptotic of

[TABLE]

The point is that we have a finite number of terms in the above sum and we can take the limit as $t\to\infty$ for each individual term. For instance we have

[TABLE]

From this we derive that for a fixed $0\leq s\leq t-1$,

[TABLE]

Now, we go back to (6.13) and notice that

[TABLE]

Now we are going to split the series from (6.13) as

[TABLE]

where $u$ is a fixed but large number such that $0<u<t-1$. For the first part of the series we have that for a fixed $u$,

[TABLE]

The second term can be controlled as follows. Take any matrix norm and estimate

[TABLE]

where $c,C>0$ are some constants independent of $u$ and $t$. Using now (6.15), (6.16) and (6.16) we can conclude that

[TABLE]

Now as the series $\sum_{s=0}^{\infty}|\frac{\delta_{s}}{\lambda^{s}}|$ is convergent in $L^{1}$ for $|\lambda|>1$ which means that we can let $u$ tend to infinity and get the conclusion that

[TABLE]

where the remainder $R_{t}$ is a random variable such that $E[|R_{t}|]\xrightarrow[t\to\infty]{}0$.

For each eigenvalue $\lambda$ with $|\lambda|=\alpha$, we can write it as $\lambda=\alpha e^{i\theta}$ for some $\theta\in[0,2\pi)$ and with this representation we now have

[TABLE]

Putting all the contributing blocks together, we get second part of the Theorem.

For the last part, namely Case III of the Theorem, for $\alpha=1$ we can simply use a multidimensional version of the CLT. For a single Jordan block we have

[TABLE]

which in distribution is the same as

[TABLE]

Using [Str10, Theorem 2.3.8] we need first to compute the covariance matrix

[TABLE]

where $\Gamma$ is the covariance matrix of $\delta_{s}$ and we use the bar here to denote the complex conjugate. Next, we write

[TABLE]

where $P_{j}$ is a matrix valued polynomial of degree $j$. The coefficient of $P_{m-1}$ is $\frac{1}{(m-1)!}L^{m-1}$ and in general we have

[TABLE]

for some constant $C>0$ which does not depend on $t$. In particular we have that

[TABLE]

The leading term in $t$ of the above expression is given by the polynomials of the largest degrees, thus

[TABLE]

To use the CLT we need to check the Lindeberg condition, namely [Str10, Equation (2.3.10)]. This is now easily done by observing that

[TABLE]

for a constant $C>0$ independent of $s$. Summing this over $s$, we see it is easy to verify now the Lindeberg condition. From this, we get the conclusion by putting together all the blocks and noticing that the eigenvalues $\lambda$ with $|\lambda|<1$ do not contribute anything if we scale $X_{t}$ by $t^{m-1/2}$.

∎

Before we jump into the proof (in the Appendix), let us make some comments on the significance of this Theorem.

6.1 Significance and interpretation of Theorem 13

We highlight some important points of the result above which might shed some light on its relevance and importance:

The first part of the Theorem is pretty straightforward and it should not come as a surprise given that we treated something like this in Theorem 5. However this is slightly stronger than the previous result because we have a complete description of the limiting distribution, not mere existence. 2. 2.

The second item is very interesting. It shows that for the case of large eigenvalues, the leading order is $t^{m-1}\alpha^{t}$. On the other hand this is not convergent if we have complex eigenvalues, because the term $Q_{max}^{t-m+1}$ oscillates. For example, in the case that all the eigenvalues are simple, this term is of the type $a\cos(t\theta)+b\sin(t\theta)$. It is also interesting to point out that for each such complex eigenvalue we obtain an oscillatory term. 3. 3.

The last item shows that we get a CLT, however this is very sensitive to the change of the matrix $A$. For the CLT, the contribution of the noise comes only through generic properties of $\gamma$, like the mean and covariance, however, in the first two items the whole noise is present in the asymptotic behavior. 4. 4.

If we keep in mind that in fact $X_{t}$ in this Theorem should be thought as $X_{t}-\bar{\sigma}$, then it becomes obvious that in general, knowing the matrix $A-\mathcal{E}$, we should be able to get statistical estimates for $\sigma$. However, as the first two items show, the noise is a contributing part of the asymptotic behavior. 5. 5.

In some cases, it might happen that the asymptotic limits in (6.9) (6.10) and (6.11) might be 0. In this case, what we can do is go back to the Jordan block and refine the estimates. 6. 6.

We can see the fragility of such types of results. A slight change in the matrix $A$ could lead to radically different behaviors for the dynamical system. The important lesson is that for understanding the limiting behavior, the distribution of the noise is an integrated part, except the rather rare case when we can see a CLT for which only the covariance matrix of the noise contributes to the limiting distribution. 7. 7.

The CLT incorporates a richer structure than possible in just standard DeGroot learning. Because of the feedback term, Case III encapsulates a basic DeGroot model. With $\alpha=1$, it is possible all the learning rates are zero, $\mathcal{E}=0$ and $A-\mathcal{E}$ in 6.1 becomes just $A$, then we have a pure noisy DeGroot model with no learning: $X_{t+1}=AX_{t}+\gamma_{t},\text{ for }t\geq 0.$ Alternatively, maybe some agents are not learning but interacting only. In that case, $\mathcal{E}$ is of a lower rank and $\mathcal{E}\neq 0$. In both situations we allow for negative weights in $A$ and in $\mathcal{E}$, as long as Case III applies and the original weights matrix $A-\mathcal{E}$ was stochastic.

6.2 Simulation to Convergence and Numerical Discussion

Figures 2 examines asymptotic case for five agents with binomial $\pm 1$ noise or uniform noise on $[-1,1]$ and the dynamical matrix given by 6.8. We consider the case of noise with independent components on the left hand column of Figure 2 and the case the noise is rotated by an orthogonal matrix in the right hand column. The simulations contain several examples, depending on the values of $\alpha$ and we plotted the pairwise distributions of the agents.

If $\alpha<1$ with binomial noise we get some Cantor like distributions. For other types of noise, this is not the case of course. Notice that the Cantor type structure appears also in the case of noise with correlated components of the noise. It is also interesting to point out that the limiting distribution heavily depends on the structure of the noise.

In the case $\alpha>1$, we have collapses of the limiting distribution. The dimension of the collapse is defined by the dimensions whose corresponding eigenvalues are $\alpha$. In figure (c) and (d), we started with a random initial value of $X_{0}$ with components uniform and independent in $[0,10]$. Notice that the limiting distribution is in fact depending not only on the noise but also on the in the initial condition, thus making this limiting case more intricate. On the other hand, the dimension of the collapse (as it is described in (6.10)) is dictated by the subspace generated by the components with eigenvalues corresponding to $\alpha$. Namely in Figure (c), the collapse is two dimensional while in Figure (d) is one dimension. This is reinforced by the scale at which the other cases live on.

The last case, which is the CLT, is $\alpha=1$ and it shows the limiting normal also lives on a lower dimensional space which is again defined by the eigenvalues of absolute values $1$. This is clarified also by the fact that the covariance matrix is defined in this fashion. For example, in the case of Figure (e) and (f), the covariance matrices of the limiting distribution are given

[TABLE]

Notice also here that in figure (e) $\lambda_{1}=1$ and $\lambda_{3}=1$ but the last component also disappears which at first site looks strange. This is in fact due to the scaling in the CLT which is much bigger than the standard CLT scaling. Ultimately, this is the reflection of the fact that the first Jordan block is bigger than the last Jordan block.

Thus, what these indicate is that the second component also vanishes in the limit, a fact which is somehow surprising but supported by the simulations as well. One last thing is that the rotation matrix applied to the noise does not have any effect on the limiting distribution for the CLT, as opposed to the other two cases because in the end, the contribution of the noise is via the covariance matrix of the noise not through the whole distribution of the noise.

When $\alpha>1$, as in some agents are positively correlated, whereas others are anti-correlated. In this case, there is no dimensional collapse. However, when we have a pure DeGroot model, Figure 2 (b), all five agents synchronize and reach stochastic consensus. In both of the Figure 2 (a) and (b), the noise is not rotated. Furthermore, in $A$ the size of the largest Jordan block is 1.

To examine a more refined Jordan block structure, we directly use 6.8 as our $A$ and vary $\lambda_{1},\lambda_{2}$ and $\lambda_{3}$. The noise though iid, can be separated in two regimes: orthogonal or identity rotations. If the noise is rotated, the dynamics changes from adding just a pure $\gamma_{t}$ term to adding term $U\,\gamma_{t}$, where $U$ is an orthogonal matrix. When $U=I$, there is no rotation as in Figure 2. The rotational cases worth examining are the fractal or cantor-like distributions that arise in the $\alpha<1$ case.

7 Conclusion

In this paper we introduce a model of interaction and learning which resembles to some extent the DeGroot model, though it is rather different in scope. The plain vanilla model is deterministic but we introduce a model with randomness and analyze convergence when the noise decays to zero in large time.

When the noise is not decaying, condition (4.4) is crucial to ensure convergence in distribution. This condition can be thought of as a stabilization feature of learning. Individuals learn with varying $A_{t}$ and $\mathcal{E}_{t}$ but these cannot change too drastically. Eventually, all agents settle down. Temporary bubbles are also possible, where agents don’t learn $\rho_{t}=0$ for shout bouts or there are periods of insanity $\rho_{t}>1$: this is an added feature of condition (4.2).

The limit theorem developed in Theorem 13 shows an intriguing phenomenon for the case of time independent matrix. Essentially, if we want to see the more refined structure of the $X_{t}$ —the opinions of the agents at time $t$—then the point is that the asymptotic behavior depends on the Jordan decomposition. In some instances, we can get a CLT. However, thinking in terms of the matrix $A$ of the dynamics, this is rather unlikely. On the other hand, if the eigenvalues stay inside the unit disk or are outside the unit disk, the main asymptotic limit depends, in fact, on the whole distribution of the noise. The more refined version of the results in Theorem 13 for the case of time varying dynamics matrix $A_{t}$ is desired, but given the fragility of the time independent case, a unitary approach seems more intricate. Of value would be a treatment in which the matrix $A_{t}$ is picked out at random with some distribution. Some of these topics appear in vastly different frameworks to ours [DF99] and [BM03].

Thus far, agents’ rules are mechanical. Future work should address the issue of rationality or an infinite number of agents. In Averaging plus Learning, individuals are boundedly rational. They use the same rule. What if the agents are strategic? In the presence of noise or disturbance, manipulation of opinion dynamics by forceful agents like an AI bot ([AOP10]) becomes an interesting but difficult question. Random dynamical systems were reviewed by [BM03, Sta09]. Our results use different techniques to study social learning. Though it must be acknowledged that recursive random dynamical systems are not new in economics, physics and computer science, their probabilistic analysis poses several challenges to researchers.