Prediction with Expert Advice under Local Differential Privacy
Ben Jacobsen, Kassem Fawaz
TL;DR
This paper advances prediction with expert advice under local differential privacy by introducing two algorithms, RW-AdaBatch and RW-Meta, which improve privacy-utility trade-offs and outperform existing methods on real-world COVID-19 hospital data.
Contribution
The paper presents two novel algorithms, RW-AdaBatch and RW-Meta, that enhance privacy and utility in LDP prediction with expert advice, including a new privacy amplification technique and expert selection method.
Findings
RW-AdaBatch provides privacy amplification with no utility loss.
RW-Meta enables private selection among complex experts without extra privacy cost.
Algorithms outperform classical and state-of-the-art DP methods on COVID-19 hospital data.
Abstract
We study the classic problem of prediction with expert advice under the constraint of local differential privacy (LDP). In this context, we first show that a classical algorithm naturally satisfies LDP and then design two new algorithms that improve it: RW-AdaBatch and RW-Meta. For RW-AdaBatch, we exploit the limited-switching behavior induced by LDP to provide a novel form of privacy amplification that grows stronger on easier data, analogous to the shuffle model in offline learning. Drawing on the theory of random walks, we prove that this improvement carries essentially no utility cost. For RW-Meta, we develop a general method for privately selecting between experts that are themselves non-trivial learning algorithms, and we show that in the context of LDP this carries no extra privacy cost. In contrast, prior work has only considered data-independent experts. We also derive formal…
Peer Reviews
Decision·ICLR 2026 Poster
- Extends the prediction-with-expert-advice framework to the local DP regime. - RW-AdaBatch builds on RW-FTPL, showing that when its predictions stay stable, several updates can be grouped together. This effectively mixes them like shuffling does in batch privacy, hence, gains privacy at little extra cost. - Introduces a new LDP setting where the meta-learner selects among data-dependent experts sharing the same privatized data stream. - In RW-Meta, the idea of reusing noise across learners is
- The work’s strength is theoretical rather than practical. Local-DP assumptions are hard to satisfy in realistic online settings, and the empirical evaluation is limited in scale and scope. - The novelty seems to be low. RW-AdaBatch’s batching idea is a reformulation of existing privacy-amplification and RW-Meta reuses known post-processing principles rather than introducing a fundamentally new privacy mechanism. - RW-Meta’s design involves maintaining and updating full covariance matrices acro
- The paper studies an important problem at the intersection of privacy and online learning - The theoretical results seem to be novel, and the experimental results are strong.
My biggest issue with this paper is its lack of clarity regarding the significance of its results. While I don't doubt that results and proofs are novel, I am having a hard time seeing what these results improve upon in private online learning. For example, a regret bound of $\frac{\sqrt{T}}{\eps}$ under local DP can be derived from Theorem 4.1 in [1] by doing the same analysis except using the full-information multiplicative weights algorithm instead of EXP2. In light of this, what are the
1. The paper investigated an interesting problem of prediction from expert advice in online learning under differential privacy. 2. The paper provided some theoretical analysis and guarantees. 3. The authors implement their methods on real data.
1. The statement of writing and notation is confusing. For example, the paper mixed the statements of LDP and CDP. In the title, the authors used LDP, but in many places, they claim CDP. For the privacy parameter, somewhere $\epsilon$ is used, and somewhere $\mu$ is used. 2. Lacks direct numerical comparisons with recent LDP online learning works [1]. 3. See more in the Questions part. [1]. Cheng, Duo, Xingyu Zhou, and Bo Ji. "Follow-the-Perturbed-Leader for Adversarial Bandits: Heavy Tails, Ro
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPrivacy-Preserving Technologies in Data · Advanced Bandit Algorithms Research · Mobile Crowdsensing and Crowdsourcing