Constant matters: Fine-grained Complexity of Differentially Private Continual Observation

TL;DR

This paper provides precise error bounds for differentially private counting algorithms under continual observation, leveraging matrix factorization and cb-norm analysis, and improves longstanding bounds in the field.

Contribution

It introduces a new matrix factorization approach for continual observation, explicitly bounds error, and improves the cb-norm bounds for the counting matrix, with applications to various problems.

Findings

Explicit error bounds for binary counting and histograms.

Improved cb-norm bounds over 28 years old results.

First lower bounds on additive error for private continual counting.

Abstract

We study fine-grained error bounds for differentially private algorithms for counting under continual observation. Our main insight is that the matrix mechanism when using lower-triangular matrices can be used in the continual observation model. More specifically, we give an explicit factorization for the counting matrix $M_\mathsf{count}$ and upper bound the error explicitly. We also give a fine-grained analysis, specifying the exact constant in the upper bound. Our analysis is based on upper and lower bounds of the {\em completely bounded norm} (cb-norm) of $M_\mathsf{count}$. Along the way, we improve the best-known bound of 28 years by Mathias (SIAM Journal on Matrix Analysis and Applications, 1993) on the cb-norm of $M_\mathsf{count}$ for a large range of the dimension of $M_\mathsf{count}$. Furthermore, we are the first to give concrete error bounds for various problems under continual observation such as binary counting, maintaining a histogram, releasing an approximately cut-preserving synthetic graph, many graph-based statistics, and substring and episode counting. Finally, we note that our result can be used to get a fine-grained error bound for non-interactive local learning {and the first lower bounds on the additive error for $(\epsilon,\delta)$-differentially-private counting under continual observation.} Subsequent to this work, Henzinger et al. (SODA2023) showed that our factorization also achieves fine-grained mean-squared error.