Differentially Private Distributed Estimation and Learning

TL;DR

This paper introduces novel differentially private distributed estimation algorithms that enable agents to estimate statistical properties while preserving privacy, with proven convergence rates and validated on real-world data.

Contribution

The paper develops new algorithms for private distributed estimation that handle dynamic networks and optimize the privacy-accuracy trade-off, extending existing methods.

Findings

Laplace noise minimizes convergence time for estimates.

Algorithms perform well on real-world power consumption data.

Trade-offs between privacy and accuracy are effectively managed.

Abstract

We study distributed estimation and learning problems in a networked environment where agents exchange information to estimate unknown statistical properties of random variables from their privately observed samples. The agents can collectively estimate the unknown quantities by exchanging information about their private observations, but they also face privacy risks. Our novel algorithms extend the existing distributed estimation literature and enable the participating agents to estimate a complete sufficient statistic from private signals acquired offline or online over time and to preserve the privacy of their signals and network neighborhoods. This is achieved through linear aggregation schemes with adjusted randomization schemes that add noise to the exchanged estimates subject to differential privacy (DP) constraints, both in an offline and online manner. We provide convergence rate analysis and tight finite-time convergence bounds. We show that the noise that minimizes the convergence time to the best estimates is the Laplace noise, with parameters corresponding to each agent's sensitivity to their signal and network characteristics. Our algorithms are amenable to dynamic topologies and balancing privacy and accuracy trade-offs. Finally, to supplement and validate our theoretical results, we run experiments on real-world data from the US Power Grid Network and electric consumption data from German Households to estimate the average power consumption of power stations and households under all privacy regimes and show that our method outperforms existing first-order, privacy-aware, distributed optimization methods.