A New Approach for Distributed Hypothesis Testing with Extensions to Byzantine-Resilience

TL;DR

This paper introduces a novel distributed hypothesis testing method where agents update beliefs without averaging, ensuring convergence to the true state even with Byzantine adversaries, under minimal assumptions.

Contribution

The paper presents a belief update rule that avoids belief-averaging and extends to Byzantine-resilient scenarios, with proven asymptotic correctness.

Findings

Agents' beliefs asymptotically concentrate on the true state.

The method is resilient to Byzantine adversaries under certain conditions.

Convergence is guaranteed with minimal assumptions on signals and network topology.

Abstract

We study a setting where a group of agents, each receiving partially informative private observations, seek to collaboratively learn the true state (among a set of hypotheses) that explains their joint observation profiles over time. To solve this problem, we propose a distributed learning rule that differs fundamentally from existing approaches, in the sense, that it does not employ any form of "belief-averaging". Specifically, every agent maintains a local belief (on each hypothesis) that is updated in a Bayesian manner without any network influence, and an actual belief that is updated (up to normalization) as the minimum of its own local belief and the actual beliefs of its neighbors. Under minimal requirements on the signal structures of the agents and the underlying communication graph, we establish consistency of the proposed belief update rule, i.e., we show that the actual beliefs of the agents asymptotically concentrate on the true state almost surely. As one of the key benefits of our approach, we show that our learning rule can be extended to scenarios that capture misbehavior on the part of certain agents in the network, modeled via the Byzantine adversary model. In particular, we prove that each non-adversarial agent can asymptotically learn the true state of the world almost surely, under appropriate conditions on the observation model and the network topology.

Equations

$$D(l_i(\cdot |{\theta }_p)||l_i(\cdot |{\theta }_q))=\sum _{w_i\in {\mathcal{S}}_i}{l}_i(w_i|{\theta }_p)\log \frac{l_i(w_i|{\theta }_p)}{l_i(w_i|{\theta }_q)}.$$

$${\pi }_{i,t+1}(\theta )=\frac{l_i(s_{i,t+1}|\theta ){\pi }_{i,t}(\theta )}{\sum _{p=1}^m{l}_i(s_{i,t+1}|{\theta }_p){\pi }_{i,t}({\theta }_p)}.$$

$${\mu }_{i,t+1}(\theta )=\frac{\min \{\{{\mu }_{j,t}(\theta ){\}}_{j\in {\mathcal{N}}_i},{\pi }_{i,t+1}(\theta )\}}{\sum _{p=1}^m\min \{\{{\mu }_{j,t}({\theta }_p){\}}_{j\in {\mathcal{N}}_i},{\pi }_{i,t+1}({\theta }_p)\}}.$$

$${\mu }_{i,t+1}(\theta )=\frac{\min \{\{{\mu }_{j,t}(\theta ){\}}_{j\in {\mathcal{M}}_{i,t}^{\theta }},{\pi }_{i,t+1}(\theta )\}}{\sum _{p=1}^m\min \{\{{\mu }_{j,t}({\theta }_p){\}}_{j\in {\mathcal{M}}_{i,t}^{{\theta }_p}},{\pi }_{i,t+1}({\theta }_p)\}}.$$

$${\mu }_{i,t+1}(\theta )={\pi }_{i,t+1}(\theta ).$$

$${\rho }_{i,t}(\theta )\triangleq \log \frac{{\pi }_{i,t}(\theta )}{{\pi }_{i,t}({\theta }^{\star })},{\lambda }_{i,t}(\theta )\triangleq \log \frac{l_i(s_{i,t}|\theta )}{l_i(s_{i,t}|{\theta }^{\star })}.$$

$${\rho }_{i,t+1}(\theta )={\rho }_{i,t}(\theta )+{\lambda }_{i,t+1}(\theta ),\forall t\in \mathbb{N}.$$

$${\rho }_{i,t}(\theta )={\rho }_{i,0}(\theta )+\sum _{k=1}^t{\lambda }_{i,k}(\theta ),\forall t\in {\mathbb{N}}_{+}.$$

$$\underset{t\to \infty }{\lim }\frac{1}{t}{\rho }_{i,t}(\theta )=-D(l_i(\cdot |{\theta }^{\star })||l_i(\cdot |\theta ))\text{almost surely}.$$

$${\mu }_{i,t}({\theta }^{\star })\ge \eta (\omega ),\forall t\ge {\bar{t}}_1(\omega ,\delta ),\forall i\in \mathcal{V}.$$

$${\mu }_{i,{\bar{t}}_1(\omega ,\delta )+1}({\theta }^{\star })$$

$$\stackrel{(b)}{\ge }\frac{\eta (\omega )}{\sum _{p=1}^m\min \{\{{\mu }_{j,{\bar{t}}_1(\omega ,\delta )}({\theta }_p){\}}_{j\in {\mathcal{N}}_i},{\pi }_{i,{\bar{t}}_1(\omega ,\delta )+1}({\theta }_p)\}}$$

$$\ge \frac{\eta (\omega )}{\sum _{p=1}^m{\pi }_{i,{\bar{t}}_1(\omega ,\delta )+1}({\theta }_p)}$$

$$\stackrel{(c)}{=}\eta (\omega ),$$

$${\bar{t}}_2^{\theta }(\omega ,\delta ,\overline{ϵ}(\omega ))\triangleq \max \{{\bar{t}}_1(\omega ,\delta ),\underset{i\in \mathcal{S}({\theta }^{\star },\theta )}{\max }\{t_i^{\theta }(\omega ,\overline{ϵ}(\omega ))\}\}.$$

$${\mu }_{i,{\bar{t}}_2+1}(\theta )$$

$$\stackrel{(b)}{\le }\frac{{\overline{ϵ}}^q(\omega )}{\sum _{p=1}^m\min \{\{{\mu }_{j,{\bar{t}}_2}({\theta }_p){\}}_{j\in {\mathcal{N}}_i},{\pi }_{i,{\bar{t}}_2+1}({\theta }_p)\}}$$

$$\le \frac{{\overline{ϵ}}^q(\omega )}{\min \{\{{\mu }_{j,{\bar{t}}_2}({\theta }^{\star }){\}}_{j\in {\mathcal{N}}_i},{\pi }_{i,{\bar{t}}_2+1}({\theta }^{\star })\}}$$

$$\stackrel{(c)}{\le }\frac{{\overline{ϵ}}^q(\omega )}{\eta (\omega )}$$

$$\stackrel{(d)}{<}{\overline{ϵ}}^{(q-1)}(\omega )\le \overline{ϵ}(\omega )<ϵ,$$

$${\mu }_{i,t}(\theta )<{\overline{ϵ}}^{(q-1)}(\omega ),\forall t\ge {\bar{t}}_2+1,\forall i\in \mathcal{S}({\theta }^{\star },\theta ).$$

$${\mathcal{L}}_1^{({\theta }^{\star },\theta )}\triangleq \{i\in \{\mathcal{V}\setminus \mathcal{S}({\theta }^{\star },\theta )\}:|{\mathcal{N}}_i\cap \mathcal{S}({\theta }^{\star },\theta )|>0\}$$

$${\mu }_{i,t}(\theta )<{\overline{ϵ}}^{(q-2)}(\omega ),\forall t\ge {\bar{t}}_2+2,\forall i\in {\mathcal{L}}_1^{({\theta }^{\star },\theta )}.$$

$${\mathcal{L}}_r^{({\theta }^{\star },\theta )}\triangleq \{i\in \mathcal{V}\setminus \{\bigcup _{c=0}^{r-1}{\mathcal{L}}_c^{({\theta }^{\star },\theta )}\}:|{\mathcal{N}}_i\cap \{\bigcup _{c=0}^{r-1}{\mathcal{L}}_c^{({\theta }^{\star },\theta )}\}|>0\}.$$

$${\mu }_{i,t}(\theta )<{\overline{ϵ}}^{(q-(r+1))}(\omega ),\forall t\ge {\bar{t}}_2+(r+1),\forall i\in {\mathcal{L}}_r^{({\theta }^{\star },\theta )},$$

$${\mu }_{j,t}(\theta )\in Conv({\Psi }_{i,t}^{\theta }),\forall j\in {\mathcal{M}}_{i,t}^{\theta },$$

$${\Psi }_{i,t}^{\theta }\triangleq \{{\mu }_{j,t}(\theta ):j\in {\mathcal{N}}_i\cap \mathcal{R}\},$$

$${t}^{\prime}(\omega)=\min\{t\in\mathbb{N}\hskip 2.84526pt{:}\hskip 2.84526pt\exists i\in\mathcal{R}\hskip 2.84526pt\textrm{with}\hskip 2.84526pt\mu_{i,t}(\theta^{\star})=0\}.\$$

$$\min \{\{{\mu }_{j,t^{\prime }(\omega )-1}({\theta }^{\star }){\}}_{j\in {\mathcal{M}}_{i,t^{\prime }(\omega )-1}^{{\theta }^{\star }}},{\pi }_{i,t^{\prime }(\omega )}({\theta }^{\star })\}>0.$$

$${\mu }_{i,t}({\theta }^{\star })\ge \eta (\omega ),\forall t\ge {\bar{t}}_1(\omega ,\delta ),\forall i\in \mathcal{R}.$$

$${\bar{t}}_2\triangleq \max \{{\bar{t}}_1(\omega ,\delta ),\underset{i\in \mathcal{S}({\theta }^{\star },\theta )\cap \mathcal{R}}{\max }\{t_i^{\theta }(\omega ,\overline{ϵ}(\omega ))\}\},$$

$$\min \{\{{\mu }_{j,{\bar{t}}_2}({\theta }^{\star }){\}}_{j\in {\mathcal{M}}_{i,{\bar{t}}_2}^{{\theta }^{\star }}},{\pi }_{i,{\bar{t}}_2+1}({\theta }^{\star })\}\ge \eta (\omega ).$$

$${\mu }_{i,t}(\theta )<{\overline{ϵ}}^{(q-1)}(\omega ),\forall t\ge {\bar{t}}_2+1,\forall i\in \mathcal{S}({\theta }^{\star },\theta )\cap \mathcal{R}.$$

$${\mathcal{L}}_1^{({\theta }^{\star },\theta )}\triangleq \{i\in \{\mathcal{V}\setminus \mathcal{S}({\theta }^{\star },\theta )\}:|{\mathcal{N}}_i\cap \mathcal{S}({\theta }^{\star },\theta )|\ge (2f+1)\}.$$

Full text

1 Introduction

Various distributed learning problems arising in social networks (such as opinion formation and spreading), and in engineering systems (such as target recognition by a group of aerial robots) can be studied under the formal framework of distributed hypothesis testing. Within this framework, a group of agents repeatedly observe certain private signals, and aim to infer the “true state of the world” that explains their joint observations. While much of the earlier work on this topic assumed the existence of a centralized fusion center for performing computational tasks [1, 2], more recent endeavors focus on a distributed setting where interactions among agents are captured by a communication graph [3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. Our work here falls in the latter class. A typical belief update rule in the distributed setting combines a local Bayesian update with a consensus-based opinion pooling of neighboring beliefs. Specifically, linear opinion pooling is studied in [3, 4, 5], whereas the log-linear form of belief aggregation is studied in the context of distributed hypothesis testing in [6, 7, 8, 9, 10], and distributed parameter estimation in [11, 12]. Notably, exponential convergence rates are achieved in [4, 6, 7, 8, 9], while a finite-time analysis is presented in [10]. Extensions to time-varying graphs have also been studied in [5, 6, 7].

In [7, Section III], the authors explain that the commonly studied linear and log-linear forms of belief aggregation are specific instances of a more general class of opinion pooling known as g-Quasi-Linear Opinion pools (g-QLOP), introduced in [13]. The main contribution of our paper is the development of a novel belief update rule that deviates fundamentally from the broad family of g-QLOP learning rules discussed above. Specifically, the learning algorithm that we propose in Section 3.1 does not rely on any linear consensus-based belief aggregation protocol. Instead, each agent maintains two sets of beliefs: a local belief that is updated in a Bayesian manner based on the private observations (without neighbor interactions), and an actual belief that is updated (up to normalization) as the minimum of the agent’s own local belief and the actual beliefs of its neighbors. In Section 6, we establish that under minimal requirements on the agents’ signal structures and the communication graph, the actual beliefs of the agents asymptotically concentrate on the true state almost surely. In Section 5, we argue that our approach works under graph-theoretic conditions that are milder than the standard assumption of strong-connectivity.

In addition to the above contribution to the distributed hypothesis testing problem, we also show in this paper that our approach is capable of handling agents that do not follow the prescribed learning algorithm. Indeed, despite the wealth of literature on distributed inference, there is limited understanding of the impact of misbehaving agents for the problem under consideration. Such agents may represent stubborn individuals, ideological extremists in the context of a social network, or model faults (either benign or malicious) in a networked control system. In the presence of such misbehaving entities, how should the remaining agents process their private observations and the beliefs of their neighbors to eventually learn the truth? To answer this question, we model misbehaving agents via the classical Byzantine adversary model, and develop a provably correct, resilient version of our proposed learning rule in Section 3.2. The only related work (that we are aware of) in this regard is reported in [9]. As we discuss in Section 3.2, our proposed approach is significantly less computationally intensive relative to those in [9]. We identify conditions on the observation model and the network structure that guarantee applicability of our Byzantine-resilient learning rule, and argue (in Section 5) that such conditions can be checked in polynomial time.

2 Model and Problem Formulation

Network Model: We consider a group of agents $\mathcal{V}=\{1,2,\ldots,n\}$ interacting over a time-invariant, directed communication graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$. An edge $(i,j)\in\mathcal{E}$ indicates that agent $i$ can directly transmit information to agent $j$. If $(i,j)\in\mathcal{E}$, then agent $i$ will be called a neighbor of agent $j$, and agent $j$ will be called an out-neighbor of agent $i$. The set of all neighbors of agent $i$ will be denoted $\mathcal{N}_{i}$. Given two disjoint sets $\mathcal{C}_{1},\mathcal{C}_{2}\subseteq{\mathcal{V}}$, we say that $\mathcal{C}_{2}$ is reachable from $\mathcal{C}_{1}$ if for every $i\in\mathcal{C}_{2}$, there exists a directed path from some $j\in\mathcal{C}_{1}$ to agent $i$ (note that $j$ will in general be a function of $i$). We will use $|\mathcal{C}|$ to denote the cardinality of a set $\mathcal{C}$.

Observation Model: Let $\Theta=\{\theta_{1},\theta_{2},\ldots,\theta_{m}\}$ denote $m$ possible states of the world; each $\theta_{i}\in\Theta$ will be called a hypothesis. Let $\mathbb{N}$ and $\mathbb{N}_{+}$ denote the set of non-negative integers and positive integers, respectively. Then at each time-step $t\in\mathbb{N}_{+}$, every agent $i\in\mathcal{V}$ privately observes a signal $s_{i,t}\in\mathcal{S}_{i}$, where $\mathcal{S}_{i}$ denotes the signal space of agent $i$. The joint observation profile so generated across the network is denoted ${s}_{t}=(s_{1,t},s_{2,t},\ldots,s_{n,t})$, where $s_{t}\in\mathcal{S}$, and $\mathcal{S}=\mathcal{S}_{1}\times\mathcal{S}_{2}\times\ldots\mathcal{S}_{n}$. The signal $s_{t}$ is generated based on a conditional likelihood function $l(\cdot|\theta^{\star})$, governed by the true state of the world $\theta^{\star}\in\Theta$. Let $l_{i}(\cdot|\theta^{\star}),i\in\mathcal{V}$ denote the $i$-th marginal of $l(\cdot|\theta^{\star})$. The signal structure of each agent $i\in\mathcal{V}$ is then characterized by a family of parameterized marginals $\{l_{i}(w_{i}|\theta):\theta\in\Theta,w_{i}\in\mathcal{S}_{i}\}$.111Whereas $w_{i}\in\mathcal{S}_{i}$ will be used to refer to a generic element of the signal space of agent $i$, $s_{i,t}$ will denote the random variable (with distribution $l_{i}(\cdot|\theta^{\star})$) that corresponds to agent $i^{\prime}$s observation at time-step $t$.

We make the following standard assumptions [3, 4, 5, 6, 7, 8, 9, 10]: (i) The signal space of each agent $i$, namely $\mathcal{S}_{i}$, is finite. (ii) Each agent $i$ has knowledge of its local likelihood functions $\{l_{i}(\cdot|\theta_{p})\}_{p=1}^{m}$, and it holds that $l_{i}(w_{i}|\theta)>0,\forall w_{i}\in\mathcal{S}_{i}$, and $\forall\theta\in\Theta$. (iii) The observation sequence of each agent is described by an i.i.d. random process over time; however, at any given time-step, the observations of different agents may potentially be correlated. (iv) There exists a fixed true state of the world $\theta^{\star}\in\Theta$ (unknown to the agents) that generates the observations of all the agents.222The approach in [6, 7] applies to a more general setting where there may not exist such a true hypothesis. Finally, we define a probability triple $(\Omega,\mathcal{F},\mathbb{P}^{\theta^{\star}})$, where $\Omega\triangleq\{\omega:\omega=(s_{1},s_{2},\ldots),\forall s_{t}\in\mathcal{S},\forall t\in\mathbb{N}_{+}\}$, $\mathcal{F}$ is the $\sigma$-algebra generated by the observation profiles, and $\mathbb{P}^{\theta^{\star}}$ is the probability measure induced by sample paths in $\Omega$. Specifically, $\mathbb{P}^{\theta^{\star}}=\prod\limits_{t=1}^{\infty}l(\cdot|\theta^{\star})$. For the sake of brevity, we will say that an event occurs almost surely to mean that it occurs almost surely w.r.t. the probability measure $\mathbb{P}^{\theta^{\star}}$.

Given the above setup, the goal of each agent in the network is to discern the true state of the world $\theta^{\star}$. The challenge associated with such a task stems from the fact that the private signal structure of any given agent is in general only partially informative. To make this notion precise, define $\Theta^{\theta^{\star}}_{i}\triangleq\{\theta\in\Theta:l_{i}(w_{i}|\theta)=l_{i}(w_{i}|\theta^{\star}),\forall w_{i}\in\mathcal{S}_{i}\}.$ In words, $\Theta^{\theta^{\star}}_{i}$ represents the set of hypotheses that are observationally equivalent to the true state $\theta^{\star}$ from the perspective of agent $i$. In general, for any agent $i\in\mathcal{V}$, we may have $|\Theta^{\theta^{\star}}_{i}|>1$, necessitating collaboration among agents. While inter-agent collaboration is implicitly assumed in the distributed hypothesis testing literature, in this paper we will also allow for potential misbehavior on the part of certain agents in the network, modeled as follows.

Adversary Model: We assume that a certain fraction of the agents are adversarial, and model their behavior based on the Byzantine fault model [14]. In particular, Byzantine agents possess complete knowledge of the observation model, the network model, the algorithms being used, the information being exchanged, and the true state of the world. Leveraging such information, adversarial agents can behave arbitrarily and in a coordinated manner, and can in particular, send incorrect, potentially inconsistent information to their out-neighbors. In terms of their distribution in the network, we will consider an $f$-local adversarial model, i.e., we assume that there are at most $f$ adversaries in the neighborhood of any non-adversarial agent.333Note that the $f$-local adversarial model assumed here is more general than the $f$-total adversarial model considered in [9], where the total number of adversaries in the entire network is upper bounded by $f$. Finally, we emphasize that the non-adversarial agents are unaware of the identities of the adversaries in their neighborhood. As is fairly standard in the distributed fault-tolerant literature [15, 16, 17, 18, 19, 20], we only assume that non-adversarial agents know the upper bound $f$ on the number of adversaries in their neighborhood. The adversarial set will be denoted by $\mathcal{A}\subset\mathcal{V}$, and the remaining agents $\mathcal{R}=\mathcal{V}\setminus\mathcal{A}$ will be called the regular agents.

Our objective in this paper will be to design a distributed learning rule that allows each regular agent $i\in\mathcal{R}$ to identify the true state of the world almost surely, despite (i) the partially informative signal structures of the agents, and (ii) the actions of any $f$-local Byzantine adversarial set. To this end, we introduce the following notion of source agents.

Definition 1**.**

(Source agents) An agent $i$ is said to be a source agent for a pair of distinct hypotheses $\theta_{p},\theta_{q}\in\Theta$, if $D(l_{i}(\cdot|\theta_{p})||l_{i}(\cdot|\theta_{q}))>0$, where $D(l_{i}(\cdot|\theta_{p})||l_{i}(\cdot|\theta_{q}))$ represents the KL-divergence between the distributions $l_{i}(\cdot|\theta_{p})$ and $l_{i}(\cdot|\theta_{q})$, and is given by:

[TABLE]

The set of all source agents for the pair $\theta_{p},\theta_{q}$ is denoted by $\mathcal{S}(\theta_{p},\theta_{q})$.444Notice that $\mathcal{S}(\theta_{p},\theta_{q})=\mathcal{S}(\theta_{q},\theta_{p})$, since $D(l_{i}(\cdot|\theta_{p})||l_{i}(\cdot|\theta_{q}))>0\iff D(l_{i}(\cdot|\theta_{q})||l_{i}(\cdot|\theta_{p}))>0$.

In words, a source agent for a pair $\theta_{p},\theta_{q}\in\Theta$ is an agent that can distinguish between the pair of hypotheses $\theta_{p},\theta_{q}$ based on its private signal structure. In our developments, we will require the following two definitions.

Definition 2**.**

($r$-reachable set) [16] For a graph $\mathcal{G}=(\mathcal{V,E})$, a set $\mathcal{C}\subseteq\mathcal{V}$, and an integer $r\in\mathbb{N}_{+}$, $\mathcal{C}$ is an $r$-reachable set if there exists an $i\in\mathcal{C}$ such that $|\mathcal{N}_{i}\setminus\mathcal{C}|\geq r$.

Definition 3**.**

(strongly $r$-robust graph w.r.t. $\mathcal{S}(\theta_{p},\theta_{q})$) For $r\in\mathbb{N}_{+}$ and $\theta_{p},\theta_{q}\in\Theta$, a graph $\mathcal{G}=(\mathcal{V,E})$ is strongly $r$-robust w.r.t. the set of source agents $\mathcal{S}(\theta_{p},\theta_{q})$, if for every non-empty subset $\mathcal{C}\subseteq\mathcal{V}\setminus\mathcal{S}(\theta_{p},\theta_{q})$, $\mathcal{C}$ is $r$-reachable.

3 Proposed Learning Rules

3.1 A Novel Belief Update Rule

In this section, we propose a novel belief update rule and discuss the intuition behind it. To introduce the key ideas underlying our basic approach, we first consider a scenario where all agents are regular, i.e., $\mathcal{R}=\mathcal{V}$. Every agent $i$ maintains and updates (at every time-step) two separate sets of belief vectors, namely, $\boldsymbol{\pi}_{i,t}$ and $\boldsymbol{\mu}_{i,t}$. Each of these vectors are probability distributions over the hypothesis set $\Theta$. We will refer to $\boldsymbol{\pi}_{i,t}$ and $\boldsymbol{\mu}_{i,t}$ as the “local” belief vector (for reasons that will soon become obvious), and the “actual” belief vector, respectively, maintained by agent $i$. The goal of each agent $i\in\mathcal{V}$ in the network will be to use its own private signals, and the information available from its neighbors, to update $\boldsymbol{\mu}_{i,t}$ sequentially so that $\lim_{t\to\infty}\mu_{i,t}(\theta^{*})=1$ almost surely. To do so, for each $\theta\in\Theta$, and at each time-step $t+1,t\in\mathbb{N}$, agent $i$ first generates $\pi_{i,t+1}(\theta)$ via a local Bayesian update rule that incorporates the private observation $s_{i,t+1}$ using $\pi_{i,t}(\theta)$ as a prior. Having generated $\pi_{i,t+1}(\theta)$, agent $i$ updates $\mu_{i,t+1}(\theta)$ (up to normalization) by setting it to be the minimum of its locally generated belief $\pi_{i,t+1}(\theta)$, and the actual beliefs $\mu_{j,t}(\theta),j\in\mathcal{N}_{i}$ of its neighbors at the previous time-step. It then reports its actual belief $\mu_{i,t+1}(\theta)$ to each of its out-neighbors.555Note that based on our algorithm, agents only exchange their actual beliefs, and not their local beliefs. The belief vectors are initialized as $\mu_{i,0}(\theta)>0,\pi_{i,0}(\theta)>0,\forall\theta\in\Theta,\forall i\in\mathcal{V}$. Subsequently, these vectors are updated at each time-step $t+1$ (where $t\in\mathbb{N}$) as follows:

• •

Step 1: Update of the local beliefs:

[TABLE]

• •

Step 2: Update of the actual beliefs:

[TABLE]

Intuition behind the learning rule: Consider the set of source agents $\mathcal{S}(\theta^{*},\theta)$ who can differentiate between a certain false hypothesis $\theta$ and the true state $\theta^{\star}$. Suppose for now that this set is non-empty. We ask: how do the agents in the set $\mathcal{S}(\theta^{\star},\theta)$ contribute to the process of collaborative learning? To answer this question, we note that the signal structures of such agents are rich enough for them to be able to eliminate $\theta$ on their own, i.e., without the support of their neighbors. Thus, the agents in $\mathcal{S}(\theta^{\star},\theta)$ should contribute towards driving the actual beliefs of their out-neighbors (and eventually, of all the agents in the set $\mathcal{V}\setminus\mathcal{S}(\theta^{\star},\theta)$) on the hypothesis $\theta$ to zero. To achieve the above objective, we are especially interested in devising a rule that ensures that the capability of the source agents $\mathcal{S}(\theta^{\star},\theta)$ to eliminate $\theta$ is not diminished due to neighbor interactions. As we shall see later, such a property will be particularly useful when certain agents in the network are adversarial. It is precisely these considerations that motivate us to employ (i) an auxiliary belief vector $\boldsymbol{\pi}_{i,t+1}$ generated via local processing (i.e., without any network influence) of the private signals, and (ii) a min-rule of the form (3). Specifically, if $i\in\mathcal{S}(\theta^{\star},\theta)$, then the sequence of local beliefs $\pi_{i,t+1}(\theta)$ will almost surely converge to [math] based on the update rule (2). Hence, for a source agent $i\in\mathcal{S(\theta^{\star},\theta)}$, $\pi_{i,t+1}(\theta)$ will play the key role of an external network-independent input in the min-rule (3). This in turn will trigger a process of belief reduction on the hypothesis $\theta$ originating at the source set $\mathcal{S(\theta^{\star},\theta)}$, and eventually propagating via the proposed min-rule to each agent in the network reachable from such a source set. The above discussion will be made precise in Section 6.

Remark 1**.**

We emphasize that the proposed min-rule (3) does not employ any form of “belief-averaging”. This feature is in stark contrast with existing approaches to distributed hypothesis testing that rely either on linear opinion pooling [3, 4, 5], or log-linear opinion pooling[11, 12, 10, 6, 7, 8, 9]. As such, the lack of linearity in our belief update rule precludes (direct or indirect) adaptation of existing analysis techniques to suit our needs. Consequently, we develop a novel sample path based proof technique in Section 6 to establish consistency of the proposed learning rule. As one of the main outcomes of this analysis, we argue that our learning rule works under graph-theoretic conditions that are in general weaker than strong-connectivity (see also Section 5).

3.2 A Byzantine-Resilient Belief Update Rule

As pointed out in the Introduction, a key benefit of our approach is that it can be extended to account for the worst-case Byzantine adversarial model described in Section 2. A standard way to analyze the impact of such adversarial agents while designing resilient distributed consensus-based protocols (for applications in consensus [16, 15], optimization [17, 18], hypothesis testing [9], and multi-agent rendezvous [21]) is to construct an equivalent matrix representation of the linear update rule that involves only the regular agents [22]. In particular, this requires expressing the iterates of a regular agent as a convex combination of the iterates of its regular neighbors, based on appropriate filtering techniques, and under certain assumptions on the network structure. While this can indeed be achieved efficiently for scalar consensus problems, for problems requiring consensus on vectors (like the belief vectors in our setting), such an approach becomes computationally prohibitive [9]. To bypass such heavy computations, and yet accommodate Byzantine attacks, we now develop a resilient version of the learning rule introduced in Section 3.1, as follows. Each agent $i\in\mathcal{R}$ acts as follows at every time-step $t+1$ (where $t\in\mathbb{N}$).

• •

Step 1: Update of the local beliefs: The local belief $\pi_{i,t+1}(\theta)$ is updated as before, based on (2).

• •

Step 2: Filtering extreme beliefs: If $|\mathcal{N}_{i}|\geq(2f+1)$, then agent $i$ performs a filtering operation as follows. It collects the actual beliefs $\mu_{j,t}(\theta)$ from each neighbor $j\in\mathcal{N}_{i}$ and sorts them from highest to lowest. It rejects the highest $f$ and the lowest $f$ of such beliefs (i.e., it throws away $2f$ beliefs in all). In other words, for each hypothesis, a regular agent retains only the moderate beliefs received from its neighbors.

• •

Step 3: Update of the actual beliefs: If $|\mathcal{N}_{i}|\geq(2f+1)$, then agent $i$ updates $\mu_{i,t+1}(\theta)$ as follows. Let the set of neighbors whose beliefs on $\theta$ are not rejected by agent $i$ (based on the previous filtering step) be denoted by $\mathcal{M}^{\theta}_{i,t}\subset\mathcal{N}_{i}$. The actual belief $\mu_{i,t+1}(\theta)$ is then updated as follows:

[TABLE]

If $|\mathcal{N}_{i}|<(2f+1)$, then agent $i$ updates $\mu_{i,t+1}(\theta)$ as follows:

[TABLE]

As with the learning rule presented in Section 3.1, agent $i$ transmits $\mu_{i,t+1}(\theta)$ to each of its out-neighbors on completion of the above steps. We will refer to the above sequence of actions as the Local-Filtering based Resilient Hypothesis Elimination (LFRHE) algorithm.

4 Main Results

In this section, we state our main results, and then comment on them in Section 5; detailed proofs of the results are presented in Section 6. Our first result establishes the correctness of the learning rule proposed in Section 3.1.

Theorem 1**.**

Suppose $\mathcal{R}=\mathcal{V}$, and that the following are true:

1. (i)

For every pair of hypotheses $\theta_{p},\theta_{q}\in\Theta$, the corresponding source set $\mathcal{S}(\theta_{p},\theta_{q})$ is non-empty. 2. (ii)

For every pair of hypotheses $\theta_{p},\theta_{q}\in\Theta$, $\mathcal{V}\setminus\mathcal{S}(\theta_{p},\theta_{q})$ is reachable from the source set $\mathcal{S}(\theta_{p},\theta_{q})$. 3. (iii)

Every agent $i\in\mathcal{V}$ has a non-zero prior belief on each hypothesis, i.e., $\pi_{i,0}(\theta)>0,\mu_{i,0}(\theta)>0$ for all $i\in\mathcal{V}$, and for all $\theta\in\Theta$.

Then, the learning rule described by equations (2) and (3) leads to collaborative learning of the true state, i.e., $\mu_{i,t}(\theta^{\star})\rightarrow 1$ almost surely $\forall i\in\mathcal{V}$.

Our second result establishes the correctness of the LFRHE algorithm proposed in Section 3.2.

Theorem 2**.**

Suppose the following are true:

1. (i)

For every pair of hypotheses $\theta_{p},\theta_{q}\in\Theta$, the graph $\mathcal{G}$ is strongly $(2f+1)$-robust w.r.t. the corresponding source set $\mathcal{S}(\theta_{p},\theta_{q})$. 2. (ii)

Each regular agent $i\in\mathcal{R}$ has a non-zero prior belief on each hypothesis, i.e., $\pi_{i,0}(\theta)>0,\mu_{i,0}(\theta)>0$ for all $i\in\mathcal{R}$, and for all $\theta\in\Theta$.

Then, the LFRHE algorithm described by equations (2), (4) and (5) leads to collaborative learning of the true state despite the actions of any $f$-local set of Byzantine adversaries, i.e., $\mu_{i,t}(\theta^{\star})\rightarrow 1$ almost surely $\forall i\in\mathcal{R}$.

Remark 2**.**

For any pair $\theta_{p},\theta_{q}\in\Theta$, notice that condition (i) of Theorem 2 (together with the definition of strong-robustness in Def. 3) requires $|\mathcal{S}(\theta_{p},\theta_{q})|\geq(2f+1)$, if $\mathcal{V}\setminus\mathcal{S}(\theta_{p},\theta_{q})$ is non-empty.

5 Discussion

(Assumptions in Theorem 1): While the first condition in Theorem 1 is a basic global identifiability condition, the second condition on the network structure is in general weaker than the standard assumption of strong-connectivity made in [3, 4, 11, 12, 10, 8]. To see why the latter statement is true, consider a scenario where $\Theta=\{\theta_{1},\theta_{2}\}$. Clearly, any agent $i\in\mathcal{S}(\theta_{1},\theta_{2})$ can discern the true state without neighbor interactions, precluding the need for incoming edges to such agents.666For the problem under consideration, the argument that the strong connectivity assumption can be relaxed applies to more general scenarios as well, where there does not necessarily exist any one agent that can identify the true state based on just its private signal structure. The underlying reason for this stems from information heterogeneity and information redundancy among agents [23], features shared by distributed estimation and detection type problems, but lacking in a standard consensus setting. Finally, the assumption of non-zero initial beliefs is fairly standard, and can be easily met by maintaining a uniform support over the hypotheses set initially.

(Assumptions in Theorem 2): The first condition in Theorem 2 blends requirements on the signal structures of the agents with those on the communication graph. To gain intuition about this condition, suppose $\Theta=\{\theta_{1},\theta_{2}\}$, and let there exist at least one agent $i\in\mathcal{V}\setminus\mathcal{S}(\theta_{1},\theta_{2})$. To enable agent $i$ to discern the truth despite potential adversaries in its neighborhood, one requires (i) redundancy in the signal structures of the agents (see Remark 2), and (ii) redundancy in the network structure to facilitate reliable information flow from $\mathcal{S}(\theta_{1},\theta_{2})$ to agent $i$. These requirements are captured by condition (i), a point made apparent in Section 6.2.

(Complexity of Checking Condition (i) in Theorem 2): Given a network of agents with associated signal structures, condition (i) in Theorem 2 can be checked in polynomial time. Specifically, for every pair $\theta_{p},\theta_{q}\in\Theta$, finding the source set $\mathcal{S}(\theta_{p},\theta_{q})$ can easily be done in polynomial time via inspection of the agents’ signal structures. For a fixed source set $\mathcal{S}(\theta_{p},\theta_{q})$, checking whether $\mathcal{G}$ is strongly $(2f+1)$-robust w.r.t. $\mathcal{S}(\theta_{p},\theta_{q})$ amounts to simulating a bootstrap percolation process on $\mathcal{G}$, with $\mathcal{S}(\theta_{p},\theta_{q})$ as the initial active set, and $(2f+1)$ as the threshold. This too can be achieved in polynomial time, as discussed in [19].

(Analogy with Distributed State Estimation): Consider the problem of collaboratively estimating the state of an LTI process based on information exchanges among agents that receive partial measurements of the state. There are natural connections between this setting, and the problem studied in this paper. For the state estimation scenario, one can fix an unstable mode of the process, and define source agents for that mode to be agents that can detect the eigenspaces associated with that mode. Interestingly, with source agents defined for each unstable mode in the manner described above, [23, Theorem 3] and [19, Theorem 7] (in the context of distributed state estimation) can be viewed as analogues of Theorem 1 and Theorem 2, respectively.

(Convergence Rate): Consider any false hypothesis $\theta\neq\theta^{\star}$. We conjecture that based on our learning rules, the actual beliefs of all the regular agents on $\theta$ will almost surely decay exponentially fast after a transient period, with the rate of decay lower bounded by $\min_{i\in\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}}D(l_{i}(\cdot|\theta^{\star})||l_{i}(\cdot|\theta))$.

6 Proofs of the Main Results

We start with the following simple lemma that characterizes the asymptotic behavior of the local belief sequences generated based on (2); we provide a proof (adapted to our notation) to keep the paper self-contained.

Lemma 1**.**

Consider an agent $i\in\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}$. Suppose $\pi_{i,0}(\theta^{\star})>0$. Then, the update rule (2) ensures that (i) $\pi_{i,t}(\theta)\rightarrow 0$ almost surely, and (ii) $\pi_{i,\infty}(\theta^{\star})\triangleq\lim_{t\to\infty}\pi_{i,t}(\theta^{\star})$ exists almost surely, and satisfies $\pi_{i,\infty}(\theta^{\star})\geq\pi_{i,0}(\theta^{\star})$.

Proof.

Pick an agent $i\in\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}$, and define:

[TABLE]

Then, based on (2), we obtain the following recursion:

[TABLE]

Rolling out the above equation over time yields:

[TABLE]

Notice that $\{\lambda_{i,t}(\theta)\}$ is a sequence of i.i.d. random variables with finite means and variances. In particular, it is easy to verify that each random variable $\lambda_{i,t}(\theta)$ has mean777More precisely, the mean here is obtained by using the expectation operator $\mathbb{E}^{\theta^{\star}}[\cdot]$ associated with the measure $\mathbb{P}^{\theta^{\star}}$. given by $-D(l_{i}(\cdot|\theta^{\star})||l_{i}(\cdot|\theta))$. Thus, based on the strong law of large numbers, we have $\frac{1}{t}\sum\limits_{k=1}^{t}\lambda_{i,k}(\theta)\rightarrow-D(l_{i}(\cdot|\theta^{\star})||l_{i}(\cdot|\theta))$ almost surely. Dividing both sides of (8) by $t$, and taking the limit as $t$ goes to infinity, we then obtain:

[TABLE]

Finally, note that based on the definition of the set $\mathcal{S}(\theta^{\star},\theta)$, $D(l_{i}(\cdot|\theta^{\star})||l_{i}(\cdot|\theta))>0$. It then follows from (9) that $\rho_{i,t}(\theta)\rightarrow-\infty$ almost surely, and hence $\pi_{i,t}(\theta)\rightarrow 0$ almost surely. For any $\theta\in\Theta^{\theta^{\star}}_{i}$, observe that $\lambda_{i,t}(\theta)=0,\forall t\in\mathbb{N}_{+}$. It then follows from (7) that for each $\theta\in\Theta^{\theta^{\star}}_{i}$, $\rho_{i,t}(\theta)=\rho_{i,0}(\theta),\forall t\in\mathbb{N}_{+}$. From the above discussion, we conclude that a limiting belief vector $\boldsymbol{\pi}_{i,\infty}$ exists almost surely, with non-zero entries corresponding to only those $\theta\in\Theta^{\theta^{\star}}_{i}$ for which $\pi_{i,0}(\theta)>0$. Part (ii) of the lemma then follows readily by noting that $\pi_{i,0}(\theta^{\star})>0$. ∎

We are now in position to prove Theorems 1 and 2.

6.1 Proof of Theorem 1

Proof.

Let $\bar{\Omega}\subseteq\Omega$ denote the set of sample paths along which for each agent $i\in\mathcal{V}$, the following hold: (i) for each $\theta\in\Theta\setminus{\Theta^{\theta^{\star}}_{i}}$, $\pi_{i,t}(\theta)\rightarrow 0$, and (ii) $\pi_{i,\infty}(\theta^{\star})\triangleq\lim_{t\to\infty}\pi_{i,t}(\theta^{\star})$ exists, and satisfies $\pi_{i,\infty}(\theta^{\star})\geq\pi_{i,0}(\theta^{\star})$. Recall that $\Theta^{\theta^{\star}}_{i}$ represents the set of hypotheses that are observationally equivalent to the true state $\theta^{\star}$ from the point of view of agent $i$. Hence, for each $\theta\in\Theta\setminus{\Theta^{\theta^{\star}}_{i}}$, we have $i\in\mathcal{S}(\theta^{\star},\theta)$. Based on the third condition in the statement of Theorem 1, and Lemma 1, we infer that $\bar{\Omega}$ has measure $1$. Thus, to prove the desired result, it suffices to confine our attention to the set $\bar{\Omega}$. Specifically, fix any sample path $\omega\in\bar{\Omega}$, and pick any $\epsilon>0$. Our goal will be to establish that along the sample path $\omega$, there exists $t(\omega,\epsilon)$ such that for all $t\geq t(\omega,\epsilon)$, $\mu_{i,t}(\theta)<\epsilon$ for all $i\in\mathcal{V}$, and for all $\theta\neq\theta^{\star}$ in the dynamics given by (3). This would be equivalent to establishing that the actual beliefs of all the agents on the true state can be made arbitrarily close to $1$ (since the proposed min-rule (3) generates a valid probability distribution over the hypothesis set at each time-step). We complete the proof in the following two steps.

Step 1**:** Lower bounding the actual beliefs on the true state: Consider the following scenario. During a transient phase, certain agents see private signals that cause them to temporarily lower their local beliefs on the true state. This in turn gets propagated via the min-rule (3) to the actual beliefs of the agents in the network. For sample paths in the set $\bar{\Omega}$, we rule out the possibility of such a transient phenomenon triggering a cascade of progressively lower beliefs on the true state. To this end, define $\gamma_{1}\triangleq\min_{i\in\mathcal{V}}\pi_{i,0}(\theta^{\star})$. Notice that $\gamma_{1}>0$ based on condition (iii) of the theorem. Given the choice of the sample path $\omega$, we notice that $\pi_{i,\infty}(\theta^{\star})$ exists for each $i\in\mathcal{V}$, and that $\pi_{i,\infty}(\theta^{\star})\geq\gamma_{1}$. Pick a small number $\delta>0$ such that $\delta<\gamma_{1}$. The following statement is then immediate. For each agent $i\in\mathcal{V}$, there exists $t_{i}(\omega,\delta)$, such that for all $t\geq t_{i}(\omega,\delta)$, $\pi_{i,t}(\theta^{\star})\geq\gamma_{1}-\delta>0$. Define $\bar{t}_{1}(\omega,\delta)\triangleq\max_{i\in\mathcal{V}}t_{i}(\omega,\delta)$. In words, $\bar{t}_{1}(\omega,\delta)$ represents the time-step beyond which the local beliefs of all the agents on the true state are lower-bounded by $\gamma_{1}-\delta$. We ask: At such a time-step, what is the lowest actual belief held by an agent on the true state? More precisely, we define $\gamma_{2}(\omega)\triangleq\min_{i\in\mathcal{V}}\{\mu_{i,\bar{t}_{1}(\omega,\delta)}(\theta^{\star})\}$. We claim $\gamma_{2}(\omega)>0$. To see this, observe that given the assumption of non-zero prior beliefs on the true state, and the structure of the proposed min-rule (3), $\gamma_{2}(\omega)$ can be [math] if and only if there exists some time-step $t^{{}^{\prime}}(\omega)\leq\bar{t}_{1}(\omega,\delta)$ such that $\pi_{i,t^{{}^{\prime}}(\omega)}(\theta^{\star})=0$, for some $i\in\mathcal{V}$. However, given the structure of the local Bayesian update rule (2), we would then have $\pi_{i,t}(\theta^{\star})=0$, for all $t\geq{t}^{{}^{\prime}}(\omega)$, contradicting the fact that $\pi_{i,t}(\theta^{\star})\geq\gamma_{1}-\delta>0,\forall t\geq\bar{t}_{1}(\omega,\delta)\geq t^{{}^{\prime}}(\omega),\forall i\in\mathcal{V}$ (the latter fact has already been established above). Having thus established that $\gamma_{2}(\omega)>0$, define $\eta(\omega)\triangleq\min\{\gamma_{1}-\delta,\gamma_{2}(\omega)\}>0$. In other words, $\eta(\omega)$ lower-bounds the lowest belief (considering both local and actual beliefs) on the true state $\theta^{\star}$ held by an agent at time-step $\bar{t}_{1}(\omega,\delta)$. We claim the following:

[TABLE]

To see why (10) is true, fix an agent $i\in\mathcal{V}$, and consider the following chain of inequalities:

[TABLE]

where $(a)$ is given by (3), $(b)$ follows from the way $\eta(\omega)$ is defined and by noting that $\pi_{i,t}(\theta^{\star})\geq\eta(\omega),\forall t\geq\bar{t}_{1}(\omega,\delta),\forall i\in\mathcal{V}$, and $(c)$ follows by noting that the local belief vectors generated via (2) (at each time-step) are valid probability distributions over the hypothesis set $\Theta$, and hence $\sum\limits_{p=1}^{m}\pi_{i,\bar{t}_{1}(\omega,\delta)+1}(\theta_{p})=1$. Since the above reasoning applies to every agent in the network, we can keep repeating it to establish (10) via induction.

Step 2**:** Upper bounding the actual beliefs on each false hypothesis: The key observation that guides the rest of the proof is as follows. While Step 1 of the proof ensures that the beliefs (both local and actual) of each agent on the true state $\theta^{\star}$ are lower-bounded by $\eta(\omega)>0$ after a finite period of time (given by $\bar{t}_{1}(\omega,\delta)$), Lemma 1 guarantees that the local beliefs on any false hypothesis $\theta$ will eventually become arbitrarily small (and in particular, smaller than $\eta(\omega)$) for each agent $i\in\mathcal{S}(\theta^{\star},\theta)$, on the sample path $\omega\in\bar{\Omega}$ under consideration. In what follows, we investigate how this impacts the actual beliefs of the agents in the network. To this end, given an $\epsilon>0$, pick a small $\bar{\epsilon}(\omega)>0$ such that $\bar{\epsilon}(\omega)<\min\{\eta(\omega),\epsilon\}$. Fix a hypothesis $\theta\neq\theta^{\star}$. By virtue of condition (i) of the theorem, we know that $|\mathcal{S}(\theta^{\star},\theta)|>0$. Let $q=d(\mathcal{G})+2$, where $d(\mathcal{G})$ represents the diameter of the graph $\mathcal{G}$. Then, based on Lemma 1, for each $i\in\mathcal{S}(\theta^{\star},\theta)$, there exists $t^{\theta}_{i}(\omega,\bar{\epsilon}(\omega))$ such that for all $t\geq t^{\theta}_{i}(\omega,\bar{\epsilon}(\omega))$, $\pi_{i,t}(\theta)\leq\bar{\epsilon}^{q}(\omega)$. Define

[TABLE]

Throughout the rest of the proof, we suppress the dependence of $\bar{t}_{2}$ on $\theta,\omega,\delta$ and $\bar{\epsilon}(\omega)$ to avoid cluttering the exposition. For any agent $i\in\mathcal{S}(\theta^{\star},\theta)$, we obtain the following chain of inequalities:

[TABLE]

where $(a)$ is given by (3), $(b)$ follows from the fact that for each $i\in\mathcal{S}(\theta^{\star},\theta)$, we have $\pi_{i,t}(\theta)\leq\bar{\epsilon}^{q}(\omega),\forall t\geq\bar{t}_{2}$, $(c)$ follows from (10) and (12), and $(d)$ follows from the way $\bar{\epsilon}$ has been chosen. In particular, note that the above chain of reasoning used to arrive at (13) applies to subsequent time-steps as well. We thus conclude:

[TABLE]

We now wish to investigate how the effect of (14) propagates through the rest of the network. If $\mathcal{V}\setminus\mathcal{S}(\theta^{\star},\theta)$ is empty, then we have reached the desired conclusion w.r.t. the false hypothesis $\theta$. If not, define

[TABLE]

as the set of immediate out-neighbors of the source set $\mathcal{S}(\theta^{\star},\theta)$. By virtue of condition (ii) of the theorem, if $\mathcal{V}\setminus\mathcal{S}(\theta^{\star},\theta)$ is non-empty, then $\mathcal{L}^{(\theta^{\star},\theta)}_{1}$ as defined above is also non-empty. Consider any agent $i\in\mathcal{L}^{(\theta^{\star},\theta)}_{1}$. By definition, agent $i$ has a neighbor in $\mathcal{S}(\theta^{\star},\theta)$ satisfying (14). This observation coupled with equations (10), (12) can be used to obtain a similar chain of inequalities as the ones featuring in (13). Specifically, we obtain:

[TABLE]

With $\mathcal{L}^{(\theta^{\star},\theta)}_{0}\triangleq\mathcal{S}(\theta^{\star},\theta)$, the above arguments can be repeated by successively defining the sets $\mathcal{L}^{(\theta^{\star},\theta)}_{r},1\leq r\leq d(\mathcal{G})$ as follows:

[TABLE]

Whenever $\mathcal{V}\setminus\{\bigcup_{c=0}^{r-1}\mathcal{L}^{(\theta^{\star},\theta)}_{c}\}$ is non-empty, condition (ii) of the theorem implies that $\mathcal{L}^{(\theta^{\star},\theta)}_{r}$ will also be non-empty. One can then easily verify via induction on $r$ that:

[TABLE]

where $1\leq r\leq d(\mathcal{G})$. Noting that $q=d(\mathcal{G})+2$, we obtain the desired result that $\mu_{i,t}(\theta)<\bar{\epsilon}(\omega)<\epsilon$, $\forall t\geq\bar{t}_{2}+d(\mathcal{G})+1,\forall i\in\mathcal{V}$. An identical argument as the one presented above can be made for each false hypothesis $\theta\neq\theta^{\star}$. This completes the proof. ∎

6.2 Proof of Theorem 2

Proof.

Consider an $f$-local adversarial set $\mathcal{A}\subset\mathcal{V}$, and let $\mathcal{R}=\mathcal{V}\setminus\mathcal{A}$. We study two separate cases.

Case 1: Consider a regular agent $i\in\mathcal{R}$ such that $|\mathcal{N}_{i}|<(2f+1)$. Based on condition (i) of the theorem, we claim that $i\in\mathcal{S}(\theta_{p},\theta_{q})$, for every pair $\theta_{p},\theta_{q}\in\Theta$. We prove this claim via contradiction. To do so, suppose there exists a pair $\theta_{p},\theta_{q}\in\Theta$, such that $i\in\mathcal{V}\setminus\mathcal{S}(\theta_{p},\theta_{q})$. As $|\mathcal{N}_{i}|<(2f+1)$, the set $\{i\}$ is clearly not $(2f+1)$-reachable (see Def. 2). Thus, $\mathcal{G}$ is not strongly $(2f+1)$-robust w.r.t. the source set $\mathcal{S}(\theta_{p},\theta_{q})$, a fact that contradicts condition (i) of the theorem. Thus, we have established that for networks satisfying condition (i) of the theorem, regular agents with fewer than $(2f+1)$ neighbors can distinguish between every pair of hypotheses. Lemma 1 then implies that such agents can discern the true state $\theta^{\star}$ by simply running the local Bayesian estimator (2), and updating actual beliefs via (5).

Case 2: We now focus only on regular agents $i$ satisfying $|\mathcal{N}_{i}|\geq(2f+1)$. For this case, the structure of the proof mirrors that of Theorem 1; we thus only elaborate on details that are specific to tackling the aspect of adversarial agents. A key property of the proposed LFRHE algorithm that will be used throughout the proof is as follows. For any $i\in\mathcal{R}$, and any $\theta\in\Theta$, the filtering operation of the LFRHE algorithm ensures that at each time-step $t\in\mathbb{N}$, we have:

[TABLE]

where

[TABLE]

and $Conv(\Psi^{\theta}_{i,t})$ is used to denote the convex hull formed by the points in the set $\Psi^{\theta}_{i,t}$. In other words, any neighboring belief (on a particular hypothesis) that agent $i$ uses in the update rule (4) lies in the convex hull of the actual beliefs of its regular neighbors (on that particular hypothesis). To see why (19) is true, partition the neighbor set $\mathcal{N}_{i}$ of a regular agent into three sets $\mathcal{U}^{\theta}_{i,t},\mathcal{M}^{\theta}_{i,t}$, and $\mathcal{J}^{\theta}_{i,t}$ as follows. Sets $\mathcal{U}^{\theta}_{i,t}$ and $\mathcal{J}^{\theta}_{i,t}$ are each of cardinality $f$, and contain neighbors of agent $i$ that transmit the highest $f$ and the lowest $f$ actual beliefs respectively, on the hypothesis $\theta$, to agent $i$ at time-step $t$. The set $\mathcal{M}^{\theta}_{i,t}$ contains the remaining neighbors of agent $i$, and is non-empty at every time-step since $|\mathcal{N}_{i}|\geq(2f+1)$. If $\mathcal{M}^{\theta}_{i,t}\cap\mathcal{A}=\emptyset$, then (19) holds trivially. Thus, consider the case when there are adversaries in the set $\mathcal{M}^{\theta}_{i,t}$, i.e., $\mathcal{M}^{\theta}_{i,t}\cap\mathcal{A}\neq\emptyset$. Given the $f$-locality of the adversarial model, and the nature of the filtering operation in the LFRHE algorithm, we infer that for each $j\in\mathcal{M}^{\theta}_{i,t}\cap\mathcal{A}$, there exist regular agents $u,v\in\mathcal{N}_{i}\cap\mathcal{R}$, such that $u\in\mathcal{U}^{\theta}_{i,t}$, $v\in\mathcal{J}^{\theta}_{i,t}$, and $\mu_{v,t}(\theta)\leq\mu_{j,t}(\theta)\leq\mu_{u,t}(\theta)$. This establishes our claim regarding equation (19).

With the above property in hand, our goal will be to now establish each of the two steps in the proof of Theorem 1. To this end, let $\bar{\Omega}\subseteq\Omega$ denote the set of sample paths along which for each agent $i\in\mathcal{R}$, the following hold: (i) for each $\theta\in\Theta\setminus{\Theta^{\theta^{\star}}_{i}}$, $\pi_{i,t}(\theta)\rightarrow 0$, and (ii) $\pi_{i,\infty}(\theta^{\star})\triangleq\lim_{t\to\infty}\pi_{i,t}(\theta^{\star})$ exists, and satisfies $\pi_{i,\infty}(\theta^{\star})\geq\pi_{i,0}(\theta^{\star})$. Based on condition (ii) of the theorem, and Lemma 1, we infer that $\bar{\Omega}$ has measure $1$. Thus, as in Theorem 1, fix a sample path $\omega\in\bar{\Omega}$, and pick $\epsilon>0$. Define $\gamma_{1}=\min_{i\in\mathcal{R}}\pi_{i,0}(\theta^{\star})$, pick a small number $\delta>0$ satisfying $\delta<\gamma_{1}$, and observe that for each agent $i\in\mathcal{R}$, there exists $t_{i}(\omega,\delta)$, such that for all $t\geq t_{i}(\omega,\delta)$, $\pi_{i,t}(\theta^{\star})\geq\gamma_{1}-\delta>0$. Define $\bar{t}_{1}(\omega,\delta)\triangleq\max_{i\in\mathcal{R}}t_{i}(\omega,\delta)$ and $\gamma_{2}(\omega)\triangleq\min_{i\in\mathcal{R}}\{\mu_{i,\bar{t}_{1}(\omega,\delta)}(\theta^{\star})\}$. As before, we claim $\gamma_{2}(\omega)>0$. To establish this claim, we need to answer the following question: Can an adversarial agent cause its out-neighbors to set their actual beliefs on $\theta^{\star}$ to be [math] by setting its own actual belief on $\theta^{\star}$ to be [math]? We argue that this is impossible under the LFRHE algorithm. By way of contradiction, suppose there exists a time-step ${t}^{\prime}(\omega)$ satisfying:

[TABLE]

In words, $t^{\prime}(\omega)$ represents the first time-step when some regular agent $i$ sets its actual belief on the true hypothesis to be zero. Clearly, $t^{\prime}(\omega)\neq 0$ based on condition (ii) of the theorem. Suppose ${t}^{\prime}(\omega)$ is some positive integer, and focus on how agent $i$ updates $\mu_{i,{t}^{\prime}(\omega)}(\theta^{\star})$ based on (4). Following similar arguments as in the proof of Theorem 1, we know that $\pi_{i,t}(\theta^{\star})>0,\forall t\in\mathbb{N},\forall i\in\mathcal{R}.$ At the same time, every belief featuring in the set $\Psi^{\theta^{\star}}_{i,{t}^{\prime}(\omega)-1}$ (as defined in equation (20)) is strictly positive based on the way ${t}^{\prime}(\omega)$ is defined. In light of the above arguments, and based on (19), (20), we infer:

[TABLE]

Thus, based on (4), we must have $\mu_{i,{t}^{\prime}(\omega)}(\theta^{\star})>0$, yielding the desired contradiction. With $\eta(\omega)\triangleq\min\{\gamma_{1}-\delta,\gamma_{2}(\omega)\}>0$, one can easily verify the following:

[TABLE]

In particular, (23) follows by (i) noting that for each $i\in\mathcal{R}$, $\pi_{i,\bar{t}_{1}(\omega,\delta)+1}(\theta^{\star})\geq\eta(\omega)$, and each belief featuring in the set $\Psi^{\theta^{\star}}_{i,\bar{t}_{1}(\omega,\delta)}$ is lower bounded by $\eta(\omega)$, (ii) leveraging (19), (20), and (iii) using a similar string of arguments as those used to arrive at (11). This completes Step 1.

To proceed with Step 2 (i.e., upper-bounding the actual beliefs on each false hypothesis), given an $\epsilon>0$, pick a small $\bar{\epsilon}(\omega)>0$ such that $\bar{\epsilon}(\omega)<\min\{\eta(\omega),\epsilon\}$. Fix a hypothesis $\theta\neq\theta^{\star}$, let $q=n+1$, and note that based on Lemma 1, for each $i\in\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}$, there exists $t^{\theta}_{i}(\omega,\bar{\epsilon}(\omega))$ such that for all $t\geq t^{\theta}_{i}(\omega,\bar{\epsilon}(\omega))$, $\pi_{i,t}(\theta)\leq\bar{\epsilon}^{q}(\omega)$. Define

[TABLE]

where we have suppressed the dependence of $\bar{t}_{2}$ on $\theta,\omega,\delta$ and $\bar{\epsilon}(\omega)$ as in the proof of Theorem 1. For any agent $i\in\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}$, observe that

[TABLE]

Combining the above with a similar line of argument as used to arrive at (13), we obtain:

[TABLE]

If $\mathcal{V}\setminus\mathcal{S}(\theta^{\star},\theta)$ is empty, then we are done. Else, define

[TABLE]

Whenever $\mathcal{V}\setminus\mathcal{S}(\theta^{\star},\theta)$ is non-empty, we claim that $\mathcal{L}^{(\theta^{\star},\theta)}_{1}$ (as defined above) is also non-empty based on condition (i) of the theorem. To see this, note that if $\mathcal{L}^{(\theta^{\star},\theta)}_{1}$ is empty, then $\mathcal{C}=\mathcal{V}\setminus\mathcal{S}(\theta^{\star},\theta)$ is not $(2f+1)$-reachable, violating the fact that $\mathcal{G}$ is strongly $(2f+1)$-robust w.r.t. $\mathcal{S}(\theta^{\star},\theta)$. We claim

[TABLE]

To verify the above claim, pick any agent $i\in\mathcal{L}^{(\theta^{\star},\theta)}_{1}\cap\mathcal{R}$. When $|\mathcal{M}^{\theta}_{i,\bar{t}_{2}+1}\cap\{\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}\}|>0$, the claim follows immediately based on (26). Consider the case when $|\mathcal{M}^{\theta}_{i,\bar{t}_{2}+1}\cap\{\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}\}|=0$. Since $i\in\mathcal{L}^{(\theta^{\star},\theta)}_{1}$, it has at least $(2f+1)$ neighbors in $\mathcal{S}(\theta^{\star},\theta)$, out of which at least $f+1$ are regular based on the $f$-locality of the adversarial model. Since the set $\mathcal{J}^{\theta}_{i,\bar{t}_{2}+1}$ has cardinality $f$, it must then be that $|\mathcal{U}^{\theta}_{i,\bar{t}_{2}+1}\cap\{\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}\}|>0$. Let $u\in\mathcal{U}^{\theta}_{i,\bar{t}_{2}+1}\cap\{\mathcal{S}(\theta^{\star},\theta)\cap\mathcal{R}\}$. Based on the way $\mathcal{M}^{\theta}_{i,\bar{t}_{2}+1}$ is defined, it must be that $\mu_{j,\bar{t}_{2}+1}(\theta)\leq\mu_{u,\bar{t}_{2}+1}(\theta)<\bar{\epsilon}^{(q-1)}(\omega),\forall j\in\mathcal{M}^{\theta}_{i,\bar{t}_{2}+1}$, where the last inequality follows from (26). This establishes our claim regarding (28). Consider the update of $\mu_{i,\bar{t}_{2}+2}(\theta)$ based on (4). In light of the above arguments (that apply identically to subsequent time-steps as well), the numerator of the fraction on the RHS of (4) is upper-bounded by $\bar{\epsilon}^{(q-1)}(\omega)$, while the denominator is lower-bounded by $\eta(\omega)$. This leads to the following conclusion:

[TABLE]

With $\mathcal{L}^{(\theta^{\star},\theta)}_{0}\triangleq\mathcal{S}(\theta^{\star},\theta)$, we recursively define the sets $\mathcal{L}^{(\theta^{\star},\theta)}_{r},1\leq r\leq(n-1)$ as follows:

[TABLE]

We complete the proof by inducting on $r$. To this end, suppose the following holds for all $0\leq r\leq(n-2)$:

[TABLE]

The claim extends to the case when $r=(n-1)$ by noting that (i) $\mathcal{L}^{(\theta^{\star},\theta)}_{(n-1)}$ is non-empty if $\mathcal{V}\setminus\{\bigcup_{c=0}^{(n-2)}\mathcal{L}^{(\theta^{\star},\theta)}_{c}\}$ is non-empty (based on condition (i) of the theorem), (ii) any agent $i\in\mathcal{L}^{(\theta^{\star},\theta)}_{(n-1)}\cap\mathcal{R}$ has at least $(2f+1)$ neighbors in the set $\bigcup_{c=0}^{(n-2)}\mathcal{L}^{(\theta^{\star},\theta)}_{c}$, of which at least $f+1$ are regular (based on the $f$-locality of the adversarial model), and (iii) using the induction hypothesis and arguments similar to those used for arriving at (29). Finally, note that the sets $\mathcal{L}^{(\theta^{\star},\theta)}_{r}$ are constructed in a way such that all agents in $\mathcal{R}$ are covered. The rest of the proof is identical to that of Theorem 1. ∎

7 Conclusion

In this paper, we introduced a distributed learning rule that differs fundamentally from those existing in the literature, in the sense, that it does not rely on any consensus-based belief aggregation protocol. Using a novel sample path based analysis technique, we established its consistency under minimal requirements on the information structures of the agents and the communication graph. We then showed that a significant benefit of the proposed learning rule is that it can be easily and efficiently modified to account for the presence of misbehaving agents in the network, modeled via the Byzantine adversary model. Ongoing work involves performing a detailed convergence rate analysis to see how such rates compare with those existing in literature. Extensions to time-varying graphs are also of interest.