Fast-MWEM: Private Data Release in Sublinear Time

TL;DR

This paper introduces a faster version of the MWEM framework for private data analysis, reducing per-iteration runtime from linear to sublinear in the number of queries, enabling more scalable private data release and linear program solving.

Contribution

The paper presents a novel lazy sampling approach for MWEM, achieving sublinear per-iteration runtime using Gumbel noise and a k-NN data structure, significantly improving scalability.

Findings

Sublinear runtime per iteration achieved.

Enhanced efficiency in private linear query release.

Improved scalability for solving linear programs under privacy constraints.

Abstract

The Multiplicative Weights Exponential Mechanism (MWEM) is a fundamental iterative framework for private data analysis, with broad applications such as answering $m$ linear queries, or privately solving systems of $m$ linear constraints. However, a critical bottleneck hindering its scalability is the $\Theta(m)$ time complexity required to execute the exponential mechanism in each iteration. We introduce a modification to the MWEM framework that improves the per-iteration runtime dependency to $\Theta(\sqrt{m})$ in expectation. This is done via a lazy sampling approach to the Report-Noisy-Max mechanism, which we implement efficiently using Gumbel noise and a $k$-Nearest Neighbor data structure. This allows for the rapid selection of the approximate score in the exponential mechanism without an exhaustive linear scan. We apply our accelerated framework to the problems of private linear query release and solving Linear Programs (LPs) under neighboring constraint conditions and low-sensitivity assumptions. Experimental evaluation confirms that our method provides a substantial runtime improvement over classic MWEM.