Privacy-Preserving Average Consensus via State Decomposition
Yongqiang Wang

TL;DR
This paper introduces a privacy-preserving average consensus method that uses state decomposition, allowing nodes to keep their initial states private while still achieving exact consensus without accuracy loss.
Contribution
A novel state decomposition approach for privacy-preserving average consensus that guarantees exact convergence and protects against internal and external inference.
Findings
Guarantees exact consensus value without error.
Prevents disclosure of initial states to neighbors.
Protects against external eavesdroppers.
Abstract
Average consensus underpins key functionalities of distributed systems ranging from distributed information fusion, decision-making, distributed optimization, to load balancing and decentralized control. Existing distributed average consensus algorithms require each node to exchange and disclose state information to its neighbors, which is undesirable in cases where the state is private or contains sensitive information. In this paper, we propose a novel approach that avoids disclosing individual state information in average consensus by letting each node decompose its state into two sub-states. For each node, one of the two sub-states involves in computation and inter-node interactions as if it were the original state, while the other sub-state interacts only with the first sub-state of the same node, being completely invisible to other nodes. The initial values of the two sub-states…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDistributed Control Multi-Agent Systems · Security in Wireless Sensor Networks · Privacy-Preserving Technologies in Data
Privacy-Preserving Average Consensus via State Decomposition
Yongqiang Wang1 The work was supported in part by the National Science Foundation under Grants 1824014 and 1738902.1Yongqiang Wang is with the Department of Electrical and Computer Engineering, Clemson University, Clemson, SC 29634, USA [email protected]
Abstract
Average consensus underpins key functionalities of distributed systems ranging from distributed information fusion, decision-making, distributed optimization, to load balancing and decentralized control. Existing distributed average consensus algorithms require each node to exchange and disclose state information to its neighbors, which is undesirable in cases where the state is private or contains sensitive information. In this paper, we propose a novel approach that avoids disclosing individual state information in average consensus by letting each node decompose its state into two sub-states. For each node, one of the two sub-states involves in computation and inter-node interactions as if it were the original state, while the other sub-state interacts only with the first sub-state of the same node, being completely invisible to other nodes. The initial values of the two sub-states are chosen randomly but with their mean fixed to the initial value of the original state, which is key to guarantee convergence to the desired consensus value. In direct contrast to differential-privacy based privacy-preserving average-consensus approaches which enable privacy by compromising accuracy in the consensus value, the proposed approach can guarantee convergence to the exact desired value without any error. Not only is the proposed approach able to prevent the disclosure of a node’s initial state to honest-but-curious neighbors, it can also provide protection against inference by external eavesdroppers able to wiretap communication links. Numerical simulations demonstrate the effectiveness of the approach and its advantages over state-of-the-art counterparts.
I Introduction
As a building block of distributed computing, average consensus has been an active research topic in computer science and optimization for decades [1, 2]. In recent years, with the advances of wireless communications and embedded systems, particularly the advent of wireless sensor networks and the Internet-of-Things, average consensus is finding increased applications in fields as diverse as automatic control, signal processing, social sciences, robotics, and optimization [3].
Conventional average consensus approaches rely on the exchange of explicit state values among neighboring nodes to reach agreement on distributed computation. Such a disclosure of state information has two potential problems. First, it breaches the privacy of participating nodes who may not want to disclose their state values containing sensitive and private information. For example, a group of individuals using average consensus to compute a common opinion may want to keep each individual’s opinion secret [4]. Another example is power systems where multiple generators want to reach agreement on cost while keeping their individual generation information private since the generation information is sensitive in bidding the right for energy selling [5]. Secondly, exchanging information through wireless or wired communications is vulnerable to eavesdroppers which try to steal information by tapping communication links. With the increased number of reported attack events, preserving data privacy has become an urgent need in many social and engineering applications.
To address the pressing need for privacy-preserving average consensus, one may resort to conventional secure multi-party computation approaches such as Yao’s Garbled Circuit [6], Shamir’s Secret Sharing algorithm [7], or many other recent advances [8]. However, such general-purpose privacy protecting approaches are both computationally and communicationally too heavy for systems with fast-evolving behaviors particularly cyber-physical systems which are subject to hard real-time constraints. For example, Yao’s Garbled Circuit has a computational latency on the order of seconds [9], whereas the tolerable computational latency is on the order of milliseconds for the real-time control of connected automated vehicles [10] and unmanned aerial vehicles [11]. Recently, several dedicated privacy-preserving solutions have been proposed for average consensus [12, 13, 14, 15, 16, 17, 18]. Most of these approaches rely on the idea of obfuscation to mask true state values by injecting carefully-designed noise on the states. One commonly used tool is differential privacy from the database literature in computer science [13, 14, 16, 17, 18]. However, obfuscation under differential privacy affects the accuracy of average consensus, preventing convergence to the exact desired value. Another tool emerged recently is the correlated-noise based obfuscation [15, 12], which can guarantee the accuracy of average consensus. Observability based approaches have also been discussed in the dynamics and control community to protect the privacy of multi-agent networks. The basic idea is to design the interaction topology to minimize the observability from a certain node, which amounts to minimizing the node’s ability to infer the initial states of other nodes in the network [19, 20, 21]. However, both the correlated-noise based and the observability based approaches are vulnerable to adversary nodes which are directly connected to a target node as well as all its neighbors [22].
To improve resilience to privacy attacks, another common approach is to employ cryptography. However, although cryptography based approaches can easily enable privacy preservation with the assistance of an aggregator or third-party [23], like in cloud-based control or computation [24, 25, 26], their extension to the completely decentralized average consensus problem in the absence of an aggregator or third-party is extremely hard due to the difficulties in decentralized key management. In fact, to our knowledge, except our recent result [27, 22], existing efforts ([28, 29]) incorporate cryptography into decentralized average consensus without giving participating nodes access to the final consensus value (note that in [29] individual participating nodes do not have access to the decryption key to decrypt the final consensus value which is obtained in the encrypted form, otherwise they will be able to decrypt intermediate computations to access other nodes’ states). Furthermore, cryptography based approaches will also significantly increase communication and computation overhead (please see e.g., [30] for detailed discussions), which is not appropriate for systems with limited resources or systems with fast evolving behaviors or subject to hard real-time constraints.
In this paper, we propose a state-decomposition based approach that can guarantee the privacy of all participating nodes in average consensus without compromising accuracy. Our basic idea is to let each node decompose its state into two sub-states with random initial values. One sub-state succeeds the role of the original state in inter-node interactions while the other sub-state only interacts with the first sub-state in the same node and thus is completely invisible to outside nodes. To ensure consensus to the right average value, the initial values of the two sub-states are randomly chosen but with their mean fixed to the initial value of the original state. Different from existing differential-privacy based approaches which sacrifice accuracy for privacy, our approach can guarantee convergence to the exact average consensus value. Unlike correlated-noise based or observability based approaches which require a node to have at least one neighbor that is not directly connected to the adversary to maintain privacy, our approach can guarantee privacy of a node even when the node and all its neighbors are directly connected to the adversary. Furthermore, the approach is completely decentralized and light-weight in computation, which makes it easily applicable to resource-restricted systems. Numerical simulation results are given to verify the results.
II Background
II-A Average Consensus
We first review the average consensus problem. Following the convention in [3], we represent a network of nodes as a graph with node set , edge set , and the adjacency matrix \mathbf{A}=\big{[}a_{ij}[k]\big{]} denoting coupling weights which satisfy if and 0 otherwise. Here is time index, denoting that could be time-varying. The set of neighbors of a node is denoted as and its cardinality is denoted as .
Throughout this paper we make the following assumption:
Assumption 1
We assume that the graph is undirected and connected, i.e., holds for all and there exists a (multi-hop) path between any pair of nodes.
We represent the state variable of a node as . For the sake of simplicity, we assume scalar states. But as commented later in Remark 5, the results are easily extendable to the case where the state is a vector. To achieve average consensus, namely convergence of all states to the average of initial values, i.e., , the update rule is formulated as [31]
[TABLE]
where resides in the range with defined as
[TABLE]
It has been well known that average consensus can be achieved if the network is connected and there exists some such that holds for all [32].
II-B Attack Model
In the paper, we consider two types of adversaries:
An honest-but-curious adversary is a node who follows all protocol steps correctly but is curious and collects received data in an attempt to learn some information about other participating nodes.
An eavesdropper is an external attacker who knows the network topology, and is able to wiretap communication links and access exchanged messages.
Generally speaking, an eavesdropper is more disruptive than an honest-but-curious node in terms of information breaches because it can snoop messages exchanged on many channels whereas the latter can only access the messages destined to it. However, an honest-but-curious node does have one piece of information that is unknown to an external eavesdropper, i.e., the internal initial state is available if node is an honest-but-curious node. We will systematically analyze the enabled privacy strength of our approach against both adversaries.
III Privacy-Preserving Approach
The key idea of our approach is a decomposition mechanism:
Decomposition Mechanism: We let each node decompose its state into two sub-states and , with the initial values and randomly chosen from the set of all real numbers under the constraint (cf. Fig. 1). The sub-state succeeds the role of the original state in inter-node interactions and it is in fact the only state value from node that can be seen by its neighbors. The other sub-state also involves in the distributed interaction by (and only by) interacting with . So the existence of is invisible to neighboring nodes of node , although it directly affects the evolution of . Taking node 1 in Fig. 1(b) for example, acts as if it were in the inter-node interactions while is invisible to nodes other than node , although it affects the evolution of . The coupling weight between the two sub-states and is symmetric and denoted as . It is a design parameter and will be elaborated later in the Weight Mechanism.
Under the state-decomposition approach, the overall dynamics become
[TABLE]
subject to .
Remark 1
Compared with (1), since every “visible” sub-state’s number of neighbors is increased by 1, the upper bound on is reduced from to with defined in (2).
In conventional average consensus algorithms, the coupling weights are required to be within , which restricts the strength of achievable privacy (as will be clear from the proof of Theorem 2). We introduce the following weight mechanism to enable strong privacy:
Weight Mechanism: For , we allow all weights and to be arbitrarily chosen from the set of all real numbers under the constraint ; For , we require that there exists a scalar such that all nonzero satisfy and all satisfy .
In the following, we first prove that under the approach, i.e., the Decomposition Mechanism and the Weight Mechanism, all states and will converge to the same average consensus value as in the conventional case (1). Then we rigorously analyze the privacy of participating nodes enabled by the proposed approach in the presence of an eavesdropper or honest-but-curious node.
Theorem 1
Under Assumption 1 and the Weight Mechanism, all sub-states in (3) converge to the average consensus value of (1), i.e.,
[TABLE]
Proof: Under the symmetric weight assumption and , one can easily obtain that for the network after decomposition, the sum of all sub-states are always time-invariant. Therefore, even the weights are allowed to be arbitrarily chosen from the set of all real numbers for , we always have
[TABLE]
Starting from , as all coupling weights compose a connected graph, the Decomposition Mechanism and the Weight Mechanism guarantee that all sub-states also compose a connected graph. According to the result on average consensus under time-varying weights [32], average consensus can still be achieved, i.e., all sub-states and will converge to , which is equal to according to (5). Further making use of the fact leads to the conclusion that all sub-states converge to .
Remark 2
For the purpose of privacy-preservation, the values of should be private to node .
Next we rigorously analyze the enabled privacy against an honest-but-curious adversary or an external eavesdropping adversary. To this end, we first give a definition of privacy.
Definition 1
The privacy of the initial value of node is preserved if an adversary cannot estimate the value of with any guaranteed accuracy.
Definition 1 requires that an adversary cannot even find a range for a private value and thus is more stringent than the privacy preservation definition considered in [12, 15] which defines privacy preservation as the inability of an adversary to uniquely determine the protected value. Next we show that even by carefully observing a node’s communication for multiple steps, an adversary cannot infer the node’s initial state with any guaranteed accuracy.
Theorem 2
Under the Decomposition Mechanism and the Weight Mechanism, an honest-but-curious node cannot infer the initial state of node with any guaranteed accuracy if node has at least one neighboring node who does not collude with node to infer (cf. Fig. 2 for an illustrative example).
Proof:
To prove that node cannot estimate with any guaranteed accuracy, we show that any arbitrary variation of is indistinguishable to node , i.e., the information accessible to node can be exactly the same even if were changed to an arbitrary value . We define the information accessible to the honest-but-curious node at iteration as I_{i}[k]=\big{\{}a_{ip}[k]\big{|}_{v_{p}\in N_{i}},\,x_{p}^{\alpha}[k]\big{|}_{v_{p}\in N_{i}}\,,x_{i}[k],\,x_{i}^{\alpha}[k],\,x_{i}^{\beta}[k],\,a_{i,\alpha\beta}[k]\big{\}}. So as time evolves, the cumulated information accessible to node can be summarized as .
To show that the privacy of the initial value can be preserved against node , i.e., node cannot estimate the value of with any guaranteed accuracy, it suffices to show that under any initial value the information accessible to node , i.e., could be exactly the same as , the cumulated information accessible to node under . This is because the only information available for node to infer the initial value is , and if could be the outcome under any initial values of , then node has no way to even find a range for the initial value . Therefore, we only need to prove that for any , could hold.
Next we show that there exist initial values of and coupling weights satisfying the requirements of the Weight Mechanism that make hold under . (Note that the alternative initial values of should guarantee that the agents still converge to the original average value after is altered to .) More specifically, under the following initial condition
[TABLE]
and coupling weights
[TABLE]
where “” represents set subtraction, it can be easily verified that holds for any . Note that the first equation in (6) is used to guarantee that the consensus value does not change under the alternative initial values and . Therefore, the honest-but-curious node cannot learn the initial state of node based on accessible information if node is also connected to another node that does not collude with node to infer (note that node is allowed to exchange information with the honest-but-curious node following the protocol, as illustrated in Fig. 2).
Remark 3
In the derivation, the choice of coupling weights , , and in (7) guarantees that starting from , all sub-states under the alternative initial value will be the same as those under the original initial value , and hence all coupling weights can be the same starting from under the two different initial value conditions. Depending on the value of , the weights , , and could be outside the range , which corroborates the necessity of allowing weights at to be arbitrarily chosen from the set of all real numbers in the Weight Mechanism. Note that since the sub-states and are also arbitrarily chosen from the set of all real numbers, the possibility of them making the denominators in (7) equal to zero is negligible.
Remark 4
Our approach can protect the privacy of node even when node and all its neighbors are directly connected to the honest-but-curious neighbor (cf. Fig. 2), which is not allowed in the privacy-preserving approaches in [12, 15]. This illustrates the advantage of the proposed state-decomposition based approach.
Similar results can be obtained for the eavesdropping adversary case:
Theorem 3
Under the Decomposition Mechanism and the Weight Mechanism, an eavesdropper cannot infer the initial state of any node with any guaranteed accuracy if node has at least one neighboring node whose interaction weight with node is inaccessible to the eavesdropper.
Proof: Following the line of reasoning in Theorem 2, we can obtain that any change in the initial value can be completely compensated by changes in , , and that are invisible to the eavesdropper. Therefore, the accessible information to the eavesdropper can be exactly the same even when were changed arbitrarily and hence the eavesdropper cannot infer the initial value of node based on accessible information.
Remark 5
The results are also applicable in the vector-state case. In fact, as long as the scalar state elements in the vector state have independent coupling weights, privacy can be naturally enabled in the vector-state case by applying results in this paper to individual scalar state elements.
IV Numerical Comparison with Existing Results
In this section, we numerically compare our state-decomposition based privacy-preserving approach with existing state-of-the-art counterparts [12, 15, 18] to confirm its advantages.
For the convenience in comparison, we represent the internal state of node as and its obfuscated version (used in state exchange with neighbors) in existing obfuscation based approaches as . We considered a network of five nodes with interaction topology and weights given in [12], which, under our formulation framework translate into and
[TABLE]
Without loss of generality, we suppose that an external eavesdropper is interested in obtaining the initial state of node 1 and constructs the following observer to infer :
[TABLE]
where the initial observer state is set to . We assume that the eavesdropper has access to all weights except which was set to a random value 0.7 in the step of the observer.
Fig. 3 gives the evolution of the network states as well as the eavesdropper’s observer state under the privacy-preserving approach in [12]. The initial states of five nodes were set to . It can be seen that although convergence to the right average value is achieved, the initial internal state of node 1, i.e., , can also be inferred by the eavesdropper’s observer .
Similar results were obtained using the approach in [15], which guaranteed accurate average consensus but not the privacy of (cf. Fig. 4).
Based on the same setup, we also simulated the proposed state-decomposition based privacy-preserving approach. The coupling weights at were randomly chosen from . The results are given in Fig. 5, which confirms that the proposed approach can protect the privacy of all nodes’ initial values against an eavesdropper while achieving accurate average consensus.
It is worth noting that although differential-privacy based approaches such as [18] can also protect the privacy of participating nodes’ initial values, they also lead to errors in the final consensus value, as confirmed by the simulation results in Fig. 6. In such approaches, due to the trade-off between privacy and accuracy, when an application calls for higher accuracy of the consensus result, the risk of disclosing one’s initial state also becomes higher.
V Conclusions
In this paper, we proposed a privacy-preserving approach for the network average consensus problem based on state decomposition. In contrast to differential-privacy based approaches which are subject to a fundamental trade-off between enabled privacy and achievable consensus accuracy, the approach is able to enable privacy preservation while guaranteeing accurate average consensus. It is also superior to correlated-noise based obfuscation approaches which can guarantee accurate average consensus but not resilience to adversaries which are directly connected to a target node as well as all its neighbors. Simulation results confirmed the theoretical predictions.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] M. H. Degroot. Reaching a consensus. Journal of the American Statistical Association , 69(345):118–121, 1974.
- 2[2] N. A. Lynch. Distributed Algorithms . Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1996.
- 3[3] R. Olfati-Saber, J. A. Fax, and R. M. Murray. Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE , 95(1):215–233, 2007.
- 4[4] J. N. Tsitsiklis. Problems in decentralized decision making and computation . Ph D thesis, 1984.
- 5[5] X. Fang, S. Misra, G. Xue, and D. Yang. Smart grid – the new and improved power grid: A survey. IEEE communications surveys & tutorials , 14(4):944–980, 2012.
- 6[6] A. C. Yao. Protocols for secure computations. In Proceedings of the 23rd Annual Symposium on Foundations of Computer Science , pages 160–164, 1982.
- 7[7] A. Shamir. How to share a secret. Communications of the ACM , pages 612–613, 1979.
- 8[8] M. Prabhakaran and A. Sahai (eds). Secure Multi-party Computation . IOS press, 2013.
