What Storage Access Privacy is Achievable with Small Overhead?

TL;DR

This paper investigates the limits of achieving storage access privacy with minimal overhead, introducing optimal differentially private schemes for ORAM, PIR, and key-value storage with logarithmic privacy budgets and constant or near-constant overhead.

Contribution

The paper provides the first asymptotically optimal constructions for differentially private storage schemes with small overhead, especially for privacy budgets of c6 = \u03a9( abla n).

Findings

Constant overhead schemes require privacy budgets c6 = a9( abla n).

Optimal schemes achieve c6 = a0( abla n) with O(1) overhead for ORAM and PIR.

Key-value storage with c6 = a0( abla n) achieves O( abla abla n) overhead using a novel hashing scheme.

Abstract

Oblivious RAM (ORAM) and private information retrieval (PIR) are classic cryptographic primitives used to hide the access pattern to data whose storage has been outsourced to an untrusted server. Unfortunately, both primitives require considerable overhead compared to plaintext access. For large-scale storage infrastructure with highly frequent access requests, the degradation in response time and the exorbitant increase in resource costs incurred by either ORAM or PIR prevent their usage. In an ideal scenario, a privacy-preserving storage protocols with small overhead would be implemented for these heavily trafficked storage systems to avoid negatively impacting either performance and/or costs. In this work, we study the problem of the best $\mathit{storage\ access\ privacy}$ that is achievable with only $\mathit{small\ overhead}$ over plaintext access. To answer this question, we consider $\mathit{differential\ privacy\ access}$ which is a generalization of the $\mathit{oblivious\ access}$ security notion that are considered by ORAM and PIR. Quite surprisingly, we present strong evidence that constant overhead storage schemes may only be achieved with privacy budgets of $\epsilon = \Omega(\log n)$. We present asymptotically optimal constructions for differentially private variants of both ORAM and PIR with privacy budgets $\epsilon = \Theta(\log n)$ with only $O(1)$ overhead. In addition, we consider a more complex storage primitive called key-value storage in which data is indexed by keys from a large universe (as opposed to consecutive integers in ORAM and PIR). We present a differentially private key-value storage scheme with $\epsilon = \Theta(\log n)$ and $O(\log\log n)$ overhead. This construction uses a new oblivious, two-choice hashing scheme that may be of independent interest.

Equations

$$\Omega \left(\frac{(1-\alpha -\delta )\cdot n}{e^{ϵ}}\right)$$

$$\Omega \left({\log }_c\left(\frac{(1-\alpha )\cdot n}{e^{ϵ}}\right)\right)$$

$$\mathrm{Pr}[\mathcal{S}(Q_1)\in S]\le e^{ϵ}\cdot \mathrm{Pr}[\mathcal{S}(Q_2)\in S]+\delta .$$

$$\mathrm{Pr}[B_c\in \mathsf{IR}(q)]=\mathrm{Pr}[\mathsf{IR}(q)\in S].$$

$$\mathrm{Pr}[\mathsf{IR}(a)\in S]\le e^{ϵ}\mathrm{Pr}[\mathsf{IR}(b)\in S]+\delta$$

$$\mathrm{Pr}[B_j\notin \mathsf{IR}(i)]\le e^{ϵ}\mathrm{Pr}[B_j\notin \mathsf{IR}(j)]+\delta =\delta$$

$$E[|\mathsf{IR}(i)|]=\sum _{j\in [n]}\mathrm{Pr}[B_j\in \mathsf{IR}(i)]\ge (1-\delta )n$$

$$\Omega \left(\frac{(1-\alpha -\delta )\cdot n}{e^{ϵ}}\right)$$

$$\mathrm{Pr}[B_j\in \mathsf{IR}(j)]\le e^{ϵ}\mathrm{Pr}[B_j\in \mathsf{IR}(i)]+\delta .$$

$$E[|\mathsf{IR}(i)|]\ge \sum _{j\ne i}\mathrm{Pr}[B_j\in \mathsf{IR}(i)]\ge (n-1)\frac{1-\alpha -\delta }{e^{ϵ}}$$

$$\frac{\mathrm{Pr}[D=Q_1\mid \mathsf{RAM}(D)=\mathcal{T}]}{\mathrm{Pr}[D=Q_2\mid \mathsf{RAM}(D)=\mathcal{T}]}\ge e^{-ϵ\cdot d(Q_1,Q_2)}\frac{\mathrm{Pr}[D=Q_1]}{\mathrm{Pr}[D=Q_2]}.$$

$$\mathrm{Pr}[$$

$$=\frac{\mathrm{Pr}[\mathsf{RAM}(D)=\mathcal{T}\mid D=Q]\mathrm{Pr}[D=Q]}{\mathrm{Pr}[\mathsf{RAM}(D)=\mathcal{T}]}$$

$$\ge e^{-ϵ\cdot d_H(Q,Q^{\prime })}\frac{\mathrm{Pr}[\mathsf{RAM}(D)=\mathcal{T}\mid D=Q^{\prime }]\mathrm{Pr}[D=Q]}{\mathrm{Pr}[\mathsf{RAM}(D)=\mathcal{T}]}$$

$$=e^{-ϵ\cdot d_H(Q,Q^{\prime })}\frac{\mathrm{Pr}[D=Q^{\prime }\mid \mathsf{RAM}(D)=\mathcal{T}]\mathrm{Pr}[D=Q]}{\mathrm{Pr}[D=Q^{\prime }]}$$

$$\mathrm{Pr}[\mathsf{RAM}(Q)=\mathcal{T}]>0.$$

$$\mathrm{Pr}[D=Q^{\prime }|\mathsf{RAM}(D)=\mathcal{T}]>0.$$

$$0$$

$$=\frac{\mathrm{Pr}[\mathsf{RAM}(D)=\mathcal{T}\mid D=Q^{\prime }]\cdot \mathrm{Pr}[D=Q^{\prime }]}{\mathrm{Pr}[\mathsf{RAM}(D)=\mathcal{T}]}$$

$$\mathrm{Pr}[$$

$$\le e^{d_H(Q,Q^{\prime })ϵ}\cdot \mathrm{Pr}[\mathsf{RAM}(D)=\mathcal{T}\mid D=Q^{\prime }]=0$$

$$\Omega \left({\log }_c\left(\frac{(1-\alpha )\cdot n}{e^{ϵ}}\right)\right)$$

$$\mathrm{Pr}[U=Q\mid \mathsf{RAM}(U)=\mathcal{T}]\ge \frac{1}{n^l}\ge \frac{(1-\alpha {)}^l}{c^{2kl}}.$$

$$\mathrm{Pr}[U=Q^{\prime }\mid \mathsf{RAM}(U)=\mathcal{T}]\ge e^{-lϵ}\frac{(1-\alpha {)}^l}{c^{2kl}}.$$

$$1$$

$$\frac{\mathrm{Pr}[\mathsf{RAM}(Q^{\prime })=\mathcal{T}]}{\mathrm{Pr}[\mathsf{RAM}(Q)=\mathcal{T}]}$$

$$\prod _{i=1}^l\frac{\mathrm{Pr}[{\mathsf{RAM}}_i(Q^{\prime })={\mathcal{T}}_i\mid {\mathsf{RAM}}_{\mathtt{pr}(Q^{\prime },i)}(Q^{\prime })={\mathcal{T}}_{\mathtt{pr}(Q^{\prime },i)}]}{\mathrm{Pr}[{\mathsf{RAM}}_i(Q)={\mathcal{T}}_i\mid {\mathsf{RAM}}_{\mathtt{pr}(Q,i)}(Q)={\mathcal{T}}_{\mathtt{pr}(Q,i)}]}$$

$$\mathrm{Pr}[$$

$$=\mathrm{Pr}[{\mathsf{RAM}}_j^O(Q)=o_j].$$

$$\mathrm{Pr}$$

$$\mathrm{Pr}[{\mathsf{RAM}}_j^D(Q)=d_j\mid {\mathsf{RAM}}_{\mathtt{pr}(Q,j)}^O=o_{\mathtt{pr}(Q,j)}].$$

$$\mathrm{Pr}[$$

$$=\mathrm{Pr}[{\mathsf{RAM}}_j^D(Q^{\prime })=d\mid {\mathsf{RAM}}_{\mathtt{pr}(Q^{\prime },j)}^O=o].$$

$$\mathrm{Pr}[{\mathsf{RAM}}_j(Q)=(d_j,o_j)\mid {\mathsf{RAM}}_{[j-1]}(Q)={\mathcal{T}}_{[j-1]}]=$$

Full text

To answer this question, we consider differential privacy access which is a generalization of the oblivious access security notion that are considered by ORAM and PIR. Quite surprisingly, we present strong evidence that constant overhead storage schemes may only be achieved with privacy budgets of $\epsilon=\Omega(\log n)$. We present asymptotically optimal constructions for differentially private variants of both ORAM and PIR with privacy budgets $\epsilon=\Theta(\log n)$ with only $O(1)$ overhead. In addition, we consider a more complex storage primitive called key-value storage in which data is indexed by keys from a large universe (as opposed to consecutive integers in ORAM and PIR). We present a differentially private key-value storage scheme with $\epsilon=\Theta(\log n)$ and $O(\log\log n)$ overhead. This construction uses a new oblivious, two-choice hashing scheme that may be of independent interest.

††journalyear: 2019††copyright: rightsretained††conference: 38th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems; June 30-July 5, 2019; Amsterdam, Netherlands††booktitle: 38th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems (PODS’19), June 30-July 5, 2019, Amsterdam, Netherlands††doi: 10.1145/3294052.3319695††isbn: 978-1-4503-6227-6/19/06††submissionid: pods74

1. Introduction

Privacy-preserving storage primitives consider the outsourcing of the storage of a database to an untrusted server with the ability for clients to retrieve database records while maintaining the privacy of the retrievals. Even though encryption can be used to hide the content of the database, patterns of how the encrypted data is accessed are revealed. The leakage of access patterns has been shown to compromise privacy in many important practical settings (IKK12, ; CGP15, ). Privacy-preserving storage primitives that guarantee retrieval privacy have been used as a critical component in many systems such as advertisement (GLM16, ), discovery of identities (BDG15, ) and publish-subscribe (CSP16, ). Therefore, a very important question involves the construction of privacy-preserving storage schemes guaranteeing retrieval privacy while ensuring that record retrieval can be performed efficiently.

A common way to formulate the privacy of retrievals is obliviousness. Obliviousness guarantees that for any two fixed sequences of record retrievals of the same length, any adversary that views all accesses to stored data cannot determine which of the two sequences induced the resulting access pattern to stored data. Obliviousness has been considered with both statistical security, providing protection from adversaries with unbounded computational resources, and computational security, where the adversary is assumed to be probabilistic polynomial time (PPT). Oblivious RAM (ORAM) and private information retrieval (PIR) are two oblivious storage primitives that have been the objective of extensive research (Goldreich87, ; CGK95, ; GO96, ; KO97, ; CMS99, ; GR05, ; GM11, ; KLO12, ; SVS13, ; PPR18, ). For ORAM, it has been shown that $\Omega(\log n)$ overhead is necessary (LN18, ). On the other hand, the best constructions for PIR require at least $\Omega(n)$ server computation over the entire outsourced database.

For integral storage infrastructure that handle many access requests per second, the increased response time and server costs caused by the usage of ORAM and PIR prevent their implementation in these important systems. In a perfect world, there would exist a storage scheme with strong privacy with only small overhead that could be implemented for these frequently accessed storage infrastructures without negatively affecting the performance and/or expenses. In our work, we address the natural question of the best privacy that may be achieved by any storage scheme with only small overhead compared to plaintext access. To our knowledge, our work is the first that considers the question of maximizing privacy for a specific efficiency goal. Previous works consider minimizing the efficiency for primitives with a specific privacy notion.

To consider this problem, we use another formulation for storage access privacy through differential privacy (DMN06, ; Dwork11, ; DR14, ). Typically, differential privacy is used in the context of privacy-preserving data analysis where global properties of the entire database are disclosed while maintaining the privacy of individual database records. In particular, differential privacy guarantees that any fixed disclosure is just as likely, usually within a multiplicative factor, regardless of whether an individual belonged to the sample population or not. Our work focuses on the notion of differentially private access which attempts to maintain privacy for individual record retrievals, but may reveal information about the entire sequence of retrievals. Roughly speaking, differential privacy guarantees that any sequence of accesses to stored data caused by the execution of a sequence of record retrievals will be just as likely, except for a multiplicative factor, caused by another sequence attained by replacing a single retrieval for a database record with a single retrieval for any other database record. Differentially private access is a generalization of oblivious access as oblivious access is achieved by fixing the differential privacy parameters to be $\epsilon=0$ and $\delta=\operatorname{\mathtt{negl}}(n)$.

With this definition, readers might now ask the following two questions. When does differentially private access make sense as a security notion? Or, might there exist a stronger security notion that is achievable with small overhead?

To answer the first question, we revisit the scenario of privacy-preserving data analysis on outsourced databases. In these scenarios, there is no sense in using storage schemes providing obliviousness to hide entire record retrieval sequences when differentially private disclosures only maintain privacy for single records. For example, suppose we wish to disclose a differentially private model trained over a sample from the database. Obliviousness would unnecessarily hide the identity of the entire retrieved sample at a high cost yet the differential privacy would guarantee the privacy about individuals in the sample. Therefore, differentially private access is the privacy notion that is complementary to differential privacy disclosures on outsourced databases. In general, differentially private access guarantees privacy for individual retrievals. For the second question, recent work by Persiano and Yeo (PY18, ) show that $\Omega(\log n)$ overhead is necessary for differentially private RAMs with parameters of $\epsilon=O(1)$ and $\delta\leq 1/3$. Therefore, the task of finding good parameters for differential privacy that provide both meaningful privacy as well as while permitting small overhead seems non-trivial.

We study differentially private variants of both PIR and ORAM. PIR enables clients to obliviously retrieve database records outsourced to a server. In PIR, both the client and the server are stateless which means that no information may be maintained between multiple record retrievals beyond the server storing the database. Since both the client and the server are stateless, PIR requires the server to perform an operation on each database record: if a record is not involved in computing the server’s reply, then it cannot be the record retrieved by the client. The majority of PIR constructions use homomorphic encryption (ABF16, ; ACL17, ) or other expensive encryptions with useful properties (KO97, ; CMS99, ; GR05, ). Some recent works consider PIR with stateful clients (PPY18b, ) or super-linear server storage (CHR17, ; BIP17, ; HOW18, ). All single-server PIR schemes consider computational security. PIR has also been studied in the multiple, non-colluding server setting where constructions only require servers to perform computation sublinear in the number of database records and provide statistical security (CGK95, ; BIM00, ).

On the other hand, ORAM allows both the client and server to be stateful and maintain information between multiple queries and allows the client to perform both record retrievals and overwrites. Additionally, ORAM allows a setup phase where the untrusted server receives an encrypted version of the database to store and the client is given a secret key. As a result of state, the server is no longer required to perform computation on every database record. With this efficiency, the majority of ORAM schemes consider only simple upload and download operations and forego the use of expensive encryption schemes of PIR that require the server to perform untrusted computation, although some previous works consider ORAM with homomorphic encryption (DDF16, ). For a database with $n$ records, it has been shown that ORAM requires overhead of $\Omega(\log n)$ records (GO96, ; LN18, ) and that $O(\log n\cdot\log\log n)$ communication suffices (PPR18, ). We also examine an extension to ORAM, which we denote as oblivious key-value storage (previously also denoted as oblivious storage), where database records are uniquely identified by keys from a large universe and clients might also attempt to retrieve a non-existent key (GMO12, ).

In our work, we study the differentially private variants of these three primitives, which we denote as differentially private information retrieval (${\mathsf{DP\text{-}\mathsf{IR}}}$), differentially private RAM (${\mathsf{DP\text{-}\mathsf{RAM}}}$) and differentially private key value storage (${\mathsf{DP\text{-}KVS}}$). We focus on the question the best privacy that can be achieved by each of these primitives that only require small overhead over plaintext access.

Our Contributions. We present both negative and positive results for both ${\mathsf{DP\text{-}\mathsf{IR}}}$ and ${\mathsf{DP\text{-}\mathsf{RAM}}}$ as well as other natural variants of these primitives.

Our lower bounds for ${\mathsf{DP\text{-}\mathsf{IR}}}$ and ${\mathsf{DP\text{-}\mathsf{RAM}}}$ apply for a wide range of privacy budgets when the database is stored in a natural encoding. In particular, we consider the balls and bins model of storage (previously considered in (GO96, ; BN16, ; CCM18, )) where each database record is considered as an opaque ball along with a key containing important metadata. While not covering all database encodings, in our opinion, the balls and bins models encompasses all natural database representations that maximize practical utility and efficiency. In particular, the contents of each record are assumed to be placed together, which is typically done to maximize data locality.

For positive results, we show that a class of simple constructions are optimal for large sets of privacy budgets. These schemes may be viewed as inserting noise into the sequence of record retrievals and/or overwrites. In particular, the real record retrievals and/or overwrites are disguised within a set that also contain fake record retrievals and/or overwrites. These schemes suffice to construct the best privacy-preserving storage protocols with very small overhead for ${\mathsf{DP\text{-}\mathsf{IR}}}$, ${\mathsf{DP\text{-}\mathsf{RAM}}}$ and ${\mathsf{DP\text{-}KVS}}$. While our constructions are simple, we draw attention to the fact that designing differentially private storage systems is delicate even with weak security notions. Some simple constructions are very appealing and, at first, seem to match our lower bounds. However, many variants of simple constructions (including our constructions) end up being completely insecure. As an example, in Section 4, we consider a simple and tempting ${\mathsf{DP\text{-}\mathsf{IR}}}$ construction and show that it only guarantees differential privacy with $\delta\rightarrow 1$, i.e., no privacy at all. Additionally, while our constructions are simple, the security proofs end up being quite involved (especially for ${\mathsf{DP\text{-}\mathsf{RAM}}}$). Furthermore, to handle the additional functionalities of ${\mathsf{DP\text{-}KVS}}$, a more complex algorithm using a novel, oblivious adaptation of the two-choice hashing scheme (Mitzenmacher01, ) is required.

DP-IR Results.

To our knowledge, previous works on ${\mathsf{DP\text{-}\mathsf{IR}}}$ consider only the multiple, non-colluding server scenarios (TDG16, ). Our work is the first to consider ${\mathsf{DP\text{-}\mathsf{IR}}}$ in the single server scenario.

We show that for any ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ that must always output the right answer and any value of $\epsilon\geq 0$, then the server must operate on all stored $n$ records. This result is very strong and somewhat surprising as there does not exist any weakening of privacy that will improve the server computational costs.

Theorem 1.1 (informal).

For any $\epsilon,\delta\geq 0$, any ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ scheme in the balls and bins model must operate on $(1-\delta)n$ records.

On the other hand, we show that this strong negative result may be circumvented by considering ${\mathsf{DP\text{-}\mathsf{IR}}}$ schemes with non-zero error probabilities $0<\alpha\leq 1$. Here, the error probability is over the internal randomness of the ${\mathsf{DP\text{-}\mathsf{IR}}}$ scheme and does not depend on the queries and/or stored data. For this case, we present the following weaker lower bound.

Theorem 1.2 (informal).

For any $\epsilon,\delta\geq 0$, any ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ scheme in the balls and bins model with error probability $\alpha>0$ must operate on

[TABLE]

records.

As we are focusing on schemes with very small overhead, the above theorem leads credulence that there might exist a ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ scheme with some small, constant error probability $\alpha>0$ that only performs $O(1)$ operations when $\epsilon=\Theta(\log n)$. We show that there exists a simple construction with these properties.

Theorem 1.3 (informal).

There exists an ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ with $\epsilon=\Theta(\log n)$ and constant error probability $\alpha>0$ that only operates on $O(1)$ records.

Due to our lower bounds, the above construction seems to be the best privacy that can be achieved by any ${\mathsf{DP\text{-}\mathsf{IR}}}$ with only constant overhead compared to plaintext storage access.

We also consider natural extensions of our ${\mathsf{DP\text{-}\mathsf{IR}}}$ with multiple, non-colluding servers. There exist several multiple-server PIR schemes (CGK95, ; BIM00, ) in literature. In our full paper, we present asymptotically tight lower bounds for the constructions in (TDG16, ).

DP-RAM Results.

For ${\mathsf{DP\text{-}\mathsf{RAM}}}$, we once again start with describing our lower bound results. Unlike ${\mathsf{DP\text{-}\mathsf{IR}}}$, there is no separation between the best lower bound for perfectly correct ${\mathsf{DP\text{-}\mathsf{RAM}}}$ and ${\mathsf{DP\text{-}\mathsf{RAM}}}$ with error probability $\alpha>0$. We now present our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ lower bound which applies for all values $0\leq\alpha\leq 1$.

Theorem 1.4 (informal).

For any $\epsilon\geq 0$, any ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ with error probability $\alpha\geq 0$ in the balls and bins model and a client that stores at most $c$ blocks must operate on

[TABLE]

records.

The above theorem essentially states that there are two ways that one can achieve very efficient ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ constructions: either increase the amount of client storage ($c$) or increase the privacy budget ($\epsilon$). In most scenarios, it is impractical to suppose that the client can store large portions of data. As our desire is to construct schemes that should be easily usable in complex systems, we minimize the requirements of our clients by assuming clients have very small amounts of storage. In (PY18, ), an $\Omega(\log(n/c))$ lower bound is given for constant $\epsilon$ and $\delta\leq 1/3$. However, their lower bound can be generalized to $\Omega(\log(n/c)/e^{\epsilon})$ for any $\epsilon\geq 0$. Their lower bound has an exponentially worse dependence on $\epsilon$ compared to our lower bound. For example, the lower bound in (PY18, ) does not preclude the existence of a ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ with $\epsilon=\Theta(\log\log n)$ and constant overhead. On the other hand, our lower bound improves the privacy budget lower bound showing that an ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ that operates on $O(1)$ records must have $\epsilon=\Omega(\log n)$. We show the existence of a constant overhead ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ with asymptotically optimal $\epsilon=\Theta(\log n)$ privacy.

Theorem 1.5 (informal).

There exists an ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ with $\epsilon=\Theta(\log n)$ that only operates on $O(1)$ records.

Once again, this construction seems to be the best privacy that can be achieved by any ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme with only $O(1)$ overhead over the baseline, unprotected storage access due to our lower bounds. Our scheme improves on prevous ${\mathsf{DP\text{-}\mathsf{RAM}}}$ schemes in (WCM16, ) which starts from Path ORAM (SVS13, ) and degrades security to improve efficiency. For their scheme to achieve even client storage of $O(\sqrt{n})$, their construction recursively stores position maps which costs both logarithmic overhead and client-to-server roundtrips. On the other hand, our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ construction uses both $O(1)$ overhead and roundtrips while achieving small client storage.

DP-KVS Results.

Finally, we consider ${\mathsf{DP\text{-}KVS}}$ which is an extension of ${\mathsf{DP\text{-}\mathsf{RAM}}}$. As a result, all ${\mathsf{DP\text{-}\mathsf{RAM}}}$ lower bounds also apply to ${\mathsf{DP\text{-}KVS}}$. Therefore, the best construction that is achievable by ${\mathsf{DP\text{-}KVS}}$ with $O(1)$ overhead could be $\epsilon=\Theta(\log n)$. Due to the difficulties of handling a larger universe of queries and possibility that clients request on-existent keys, we present a construction with slightly worse than constant overhead. In addition, we can only achieve privacy in the approximate differential privacy framework.

Theorem 1.6 (informal).

There exists a ${\epsilon\mathsf{\text{-}DP\text{-}KVS}}$ with $\epsilon=\Theta(\log n)$ that operates on $O(\log\log n)$ records.

While the scheme has non-constant overhead, the $O(\log\log n)$ overhead is exponentially smaller than the best oblivious key-value storage schemes based on ORAMs.

To construct our ${\mathsf{DP\text{-}KVS}}$ scheme, we present an improved, oblivious variant of the power of two choices hashing scheme (Mitzenmacher01, ) that may be of separate, independent interest. Traditional power of two choices hashing guarantees that bins do not exceed $O(\log\log n)$ items except with probability negligible in $n$. Our desired oblivious variant must hide the sizes of bins. One way to hide bin sizes is to pad all bins with dummy items up to a maximum. This technique requires padding all $n$ bins to $O(\log\log n)$ items meaning $O(n\log\log n)$ storage. By using a tree-like structure to allow bins to share storage, we present a new scheme using $O(n)$ storage.

Related Work. ${\mathsf{DP\text{-}\mathsf{RAM}}}$ was considered previously in (WCM16, ) which present a construction based on Path ORAM (SVS13, ). However, their scheme requires recursively stored position maps which requires $\Theta(\log n)$ client-to-server roundtrips to get client storage of even $O(\sqrt{n})$. We present ${\mathsf{DP\text{-}\mathsf{RAM}}}$ schemes that only require $O(1)$ overhead with small client storage. A construction of ${\mathsf{DP\text{-}\mathsf{IR}}}$ in the multiple, non-colluding server scenario was considered in (TDG16, ), which we show is optimal for certain parameters. Lower bounds for ${\mathsf{DP\text{-}\mathsf{RAM}}}$ have been considered in (PY18, ), which are stronger and weaker in different dimensions. Their lower bound has an exponentially worse dependence on the privacy budget, $\epsilon$. On the other hand, their lower bound applies to general storage encodings whereas our work only apply to the balls and bins model. A variant of ${\mathsf{DP\text{-}\mathsf{RAM}}}$ that only maintains privacy for database record insertions is considered in (CCM18, ). A privacy notion stronger than obliviousness is considered in (KKN17, ) where accesses are protected using obliviousness while the number of accesses is protected using differential privacy. Obliviousness has been considered for other problems such as sorting (Batcher68, ; AKS83, ) and shuffling (Waksman68, ; OGT14, ; PPY18c, ). The problem of relaxing the security notion of obliviousness to differential privacy has also been studied in the context of multi-party protocols. A simple multi-party protocol for single-bit inputs is shown in (NIPS2015_6004, ) that maximizes the accuracy for any privacy budget. The multi-party computation problem for larger input sizes is studied in (MG17, ). Additionally, the problem of differentially private disclosure of specific analysis such as subspace clustering (NIPS2015_5991, ), deep learning (ACG16, ) and many others have also been studied where the database is assumed to be stored on a trusted server.

2. Definitions

We suppose the database $D$ contains $n$ records denoted $B_{1},\ldots,B_{n}$. We will interchangeably use the terms records and blocks. We will refer to a query as a single operation involving either a record retrieval or overwrite and a query sequence as a list consisting of record retrievals and/or overwrites. We use $[n]$ to refer to the set $\{1,\ldots,n\}$. We refer to $\mathcal{Q}$ as the space of all possible queries and let $Q_{1},Q_{2}\in\mathcal{Q}^{l}$ be two query sequences of length $l$. We define the Hamming distance between $Q_{1}$ and $Q_{2}$, which we denote by $d(Q_{1},Q_{2})$, as the number of queries where $Q_{1}$ and $Q_{2}$ differ.

2.1. Storage Primitives

The information retrieval ($\mathsf{IR}$) primitive stores a database $D$ of $n$ equal sized records where only record retrievals are allowed. The initialization of $\mathsf{IR}$ consists of the server receiving $D=(B_{1},\ldots,B_{n})$ and simply processing and storing $D$. The server is only allowed to store the database and may not keep any other information between multiple queries. Similarly, the client is stateless and may not use any storage between multiple queries. A query to $\mathsf{IR}$ is described using an integer $q\in[n]$, which is interpreted as retrieving record $B_{q}$. We denote two query sequences $Q_{1},Q_{2}\in[n]^{l}$ as adjacent if their Hamming distance is exactly 1. That is, $Q_{1}$ and $Q_{2}$ retrieve a different record at exactly one query.

The random access memory ($\mathsf{RAM}$) primitive will store a database $D$ of $n$ equal sized records where both record retrievals and overwrites are permitted. The initialization of $\mathsf{RAM}$ consists of a setup phase consisting of a protocol run between the client and the server. The client will receive a private key, process $D$ using the private key and send the processed database to the server to store. Both the client and server may be stateful and maintain information between multiple queries. A query to $\mathsf{RAM}$ is a pair $q=(i,\text{\sf op})$ where $i\in[n]$ refers to record $B_{i}$ and $\text{\sf op}\in\{\text{\sf read},\text{\sf write}\}$ describes whether the query is a retrieval or overwrite. Two query sequences $Q_{1},Q_{2}\in([n]\times\{\text{\sf read},\text{\sf write}\})^{l}$ are adjacent if their Hamming distance is exactly 1. That is, at exactly one query, $Q_{1}$ and $Q_{2}$ operate on a different record and/or perform a different operation.

Finally, the key-value storage ($\mathsf{KVS}$) primitive is an extension to $\mathsf{RAM}$. Queries to $\mathsf{KVS}$ consist of a pair $q=(k,\text{\sf op})$ where $k\in U$ is the key and $U$ is universe of all keys and $\text{\sf op}\in\{\text{\sf read},\text{\sf write}\}$ refers to whether the query is a retrieval or overwrite. Unlike $\mathsf{RAM}$, the universe of keys is very large and might be exponentially larger than the number of operations that will be performed. Furthermore, a retrieval operation $q=(k,\text{\sf read})$ may request a key $k$ that has never been previoulsy inserted into the storage protocol. In this case, the $\mathsf{KVS}$ protocol should output $\perp$. Identical to $\mathsf{RAM}$, two query sequences $Q_{1},Q_{2}\in(U\times\{\text{\sf read},\text{\sf write}\})^{l}$ are adjacent if there exists exactly one query that operates on a different key and/or performs a different operation.

We note that $\mathsf{IR}$, $\mathsf{RAM}$ and $\mathsf{KVS}$ are the most studied storage primitives to provide oblivious access. There are many other extensions to these primitives that have also been studied (GMO12, ; MBN15, ; BCR16, ; CLT16, ; CGL17, ; CHR17, ; BIP17, ; HOW18, ).

2.2. Differentially Private Access to Data

Our privacy notion for a storage primitive $\mathcal{S}$ storing a database $D=(B_{1},\ldots,B_{n})$ with a query space $\mathcal{Q}$ will consider the random variable of the view of the adversarial server for a query sequence $Q\in\mathcal{Q}^{l}$ of length $l$. With a slight abuse of notation, we refer to the transcript, $\mathcal{S}(Q)$, as the random variable of the adversary’s view on query sequence $Q$. The transcript contains all movement of records performed by the server, as well as all, possibly encrypted, records that have been uploaded and downloaded and the initial database.

Definition 2.1 (Differentially Private Access).

Let $\mathcal{S}$ be a storage primitive with query space $\mathcal{Q}$. $\mathcal{S}$ provides $(\epsilon,\delta)$-differentially private access if for all pairs of query sequences $Q_{1},Q_{2}\in\mathcal{Q}^{l}$ of length $l$ that are adjacent, that is $d(Q_{1},Q_{2})=1$, and for any subset $S$ of the set of possible views of the adversary, then the following holds:

[TABLE]

The $\epsilon$ parameter is referred to as the privacy budget. When $\delta=0$, the above definition is referred to as pure differential privacy and the $\delta$ is typically dropped from notation. We denote $\mathsf{IR}$, $\mathsf{RAM}$ and $\mathsf{KVS}$ primitives that provide pure differentially private access with privacy budget $\epsilon$ as ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$, ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ and ${\epsilon\mathsf{\text{-}DP\text{-}KVS}}$. When $\delta>0$, the definition is referred to as approximate differential privacy, which is a weakening of pure differential privacy. We denote $\mathsf{IR}$, $\mathsf{RAM}$ and $\mathsf{KVS}$ primitives that provide differentially private access with parameters $\epsilon$ and $\delta$ as ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$, ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ and ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}KVS}}}}$.

3. Lower Bounds

In this section, we present negative results about ${\mathsf{DP\text{-}\mathsf{IR}}}$ and ${\mathsf{DP\text{-}\mathsf{RAM}}}$ in the single-server setting. As ${\mathsf{DP\text{-}KVS}}$ is an extension of ${\mathsf{DP\text{-}\mathsf{RAM}}}$ with more functionality requirements, all ${\mathsf{DP\text{-}\mathsf{RAM}}}$ lower bounds apply directly to ${\mathsf{DP\text{-}KVS}}$.

3.1. Balls and Bins Model

Our lower bounds are presented in the balls and bins model of data manipulation. The $n$ database records are treated as immutable, opaque balls. Each ball is associated with a mutable key containing metadata about the record. The formal definition involves a ${\sf{server}}_{m}$ with storage for $m$ balls and a ${\sf{client}}_{c}$ with storage for $c$ balls.

Definition 3.1 (Balls and Bins Model).

A ${\sf{client}}_{c}$ and ${\sf{server}}_{m}$ operate in the balls and bins model if all client memory is initially empty and client-server interactions are restricted to the following:

1. (1)

Download ball from ${\sf{server}}_{m}$ to ${\sf{client}}_{c}$. For some $i\in[m]$ and $j\in[c]$, store the block at address $i$ at ${\sf{server}}_{m}$ in address $j$ at ${\sf{client}}_{c}$. 2. (2)

Upload ball from ${\sf{client}}_{c}$ to ${\sf{server}}_{m}$. For some $i\in[m]$ and $j\in[c]$, store the block at address $j$ at ${\sf{client}}_{c}$ in address $i$ at ${\sf{server}}_{m}$.

The above definition assumes a passive server acting only as storage. As a result, only lower bounds on communication can be proven directly. However, for algorithms with general computation by the server, lower bounds on communication in the balls and bins model may be modified to provide lower bounds on server computation. The modification simply views the transcript as the balls that must be operated on by the server.

Discussion about storage model.

The balls and bins model does not include all possible encodings of databases that may be stored. In particular, we assume that database records are all stored together and the contents do not emit special properties. In general, related information should be stored in nearby memory locations so that all required data can be found with the minimal number of cache misses. Therefore, assuming database records are stored together captures real world scenarios. In addition, opaque balls rule out bothersome corner cases where non-trivial lower bounds cannot hold such as when records are dependent and record contents may be generated using superficial methods.

3.2. DP-IR

We prove our lower bounds directly for approximately differential privacy (in this case, $\delta\geq 0$). However, one can interpret the results for pure differential privacy by setting $\delta=0$.

To prove our lower bounds, we first define the notion of transcripts. For a single-server information retrieval protocol $\mathsf{IR}$ in the balls and bins model, the transcript $\mathsf{IR}(i)$ is the random variable denoting the set of blocks requested when retrieving $B_{i}$. The query algorithm of an IR algorithm in the balls and bins model only issues download commands to the server.

The main observation to prove the above theorem is that the initialization phase of ${\mathsf{DP\text{-}\mathsf{IR}}}$ is public which implies that the adversary has knowledge of the identity of each record. By the restriction on correctness, the queried record must always be retrieved. Using the privacy requirement, it turns out all other blocks must also be retrieved. We summarize this in the following main lemma.

Lemma 3.2.

Let $\mathsf{IR}$ be an ${{(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}}$ scheme in the balls and bins model. For any $a,b,c\in[n]$,

1. (1)

$\Pr[B_{c}\in\mathsf{IR}(a)]\leq e^{\epsilon}\Pr[B_{c}\in\mathsf{IR}(b)]+\delta.$** 2. (2)

$\Pr[B_{c}\notin\mathsf{IR}(a)]\leq e^{\epsilon}\Pr[B_{c}\notin\mathsf{IR}(b)]+\delta.$**

Proof.

Define $S\subseteq{\mathsf{Range}}(\mathsf{IR})$ to be all transcripts where the block $B_{c}$ is downloaded. So, for any query $q$,

[TABLE]

By $(\epsilon,\delta)$-DP, we know

[TABLE]

completing the first point. The second point follows identically if we choose $S\subseteq{\mathsf{Range}}(\mathsf{IR})$ to be all transcripts where block $B_{c}$ is not downloaded. ∎

Using the above lemma, we present very strong negative results for ${\mathsf{DP\text{-}\mathsf{IR}}}$ that are errorless. That is, they always return the correct record. We show the following result:

Theorem 3.3.

If $\mathsf{IR}$ is an ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ scheme in the balls and bins model then $\mathsf{IR}$ performs at least $(1-\delta)n$ operations in expectation.

Proof.

By Lemma 3.2, we know that

[TABLE]

since $\mathsf{IR}$ is errorless and $\Pr[B_{j}\notin\mathsf{IR}(j)]=0$. Therefore, we know that $\Pr[B_{j}\in\mathsf{IR}(i)]\geq 1-\delta$ and

[TABLE]

completing the proof. ∎

For algorithms with general server computation, we can the interpret the above results as a lower bound on server computation. This result is extremely strong since $n$ server operations must be executed even when increasing the privacy budget. Therefore, the relaxation to ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ does not result in any gain compared to PIR. For ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$, one could increase $\delta$ to decrease costs. However, typical privacy requires $\delta=\operatorname{\mathtt{negl}}(n)$ resulting in almost no gain.

To circumvent this result, we move to the case where ${\mathsf{DP\text{-}\mathsf{IR}}}$ has a non-zero error rate $0<\alpha\leq 1$. That is, ${\mathsf{DP\text{-}\mathsf{IR}}}$ will only retrieve the desired record with probability $1-\alpha$ depending only on the internal randomness of ${\mathsf{DP\text{-}\mathsf{IR}}}$. Our hope is that a very small $\alpha$ may significantly improve efficiency of ${\mathsf{DP\text{-}\mathsf{IR}}}$ and bypass this negative result. We show:

Theorem 3.4.

If $\mathsf{IR}$ is an ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ scheme in the balls and bins model with error probability $\alpha>0$, then $\mathsf{IR}$ performs

[TABLE]

operations in expectation.

Proof of Theorem 3.4.

Since IR has error probability at most $\alpha$, $\Pr[B_{j}\in\mathsf{IR}(j)]\geq 1-\alpha$, for all $j\in[n]$. By Lemma 3.2, for all $j\neq i$,

[TABLE]

Equivalently, this means that $\Pr[B_{j}\in\mathsf{IR}(i)]\geq\frac{1-\alpha-\delta}{e^{\epsilon}}$. So,

[TABLE]

that yields the theorem. ∎

For any constant error $\alpha>0$ and typical privacy budgets of $\epsilon=\Theta(1)$, errorless ${\mathsf{DP\text{-}\mathsf{IR}}}$ schemes require $\Omega(n)$ server operations. However, it seems possible to bypass the first negative result with very small error. The above lower bound does not preclude the existence of a ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ with $\epsilon=\Omega(\log n)$ and small error probability $\alpha>0$ that only requires a constant number of server operations. In Section 5, we present an ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ scheme that uses $O(1)$ communication when $\epsilon=O(\log n)$. It turns out this ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ scheme asymptotically matches the lower bound for all values of $\epsilon\geq 0$.

We also extend our negative results for ${\mathsf{DP\text{-}\mathsf{IR}}}$ in Section C where we present lower bounds when outsourcing storage to multiple, non-colluding servers.

3.3. DP-RAM

We now move to our negative results for ${\mathsf{DP\text{-}\mathsf{RAM}}}$. Proving lower bounds for ${\mathsf{DP\text{-}\mathsf{RAM}}}$ is more challenging than ${\mathsf{DP\text{-}\mathsf{IR}}}$ due to the private setup phase and the client’s private memory. We can no longer directly bound the probability that blocks need to be retrieved and, instead, examine the transcripts seen by the adversary. Let us fix any transcript $\mathcal{T}$ that has non-zero probability of being viewed by the adversary on any query sequence $Q$ of length $l$. We show that every other query sequence of length $l$ must also induce $\mathcal{T}$ with non-zero probability. In fact, the probabilities that any two fixed query sequences induce $\mathcal{T}$ as the view of the adversary are strong related by their Hamming distance and the privacy budget, $\epsilon$, as described in the following:

Lemma 3.5.

Let $\mathsf{RAM}$ be a ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ scheme for any $\epsilon\geq 0$. For every distribution $\mathcal{D}$ on query sequences of length $l$, random variable $\operatorname{\text{\sf D}}\sim\mathcal{D}$ and for any two fixed sequences $Q_{1},Q_{2}$ such that $\Pr[D=Q_{2}]>0$, then

[TABLE]

Proof.

By Bayes’ law, we have

[TABLE]

giving us our result. ∎

Next, we prove that if a query sequence has positive probability, then it stays so even after seeing a transcript. This holds for all a priori query sequences distributions $\mathcal{D}$.

Lemma 3.6.

Let $\mathsf{RAM}$ be an ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ and let $\mathcal{T}$ be a transcript for which there exists at least one query sequence $Q$ such that

[TABLE]

Then for all distributions $\operatorname{\text{\sf D}}$ on the set of query sequences and for all $Q^{\prime}$ such that $\Pr[\operatorname{\text{\sf D}}=Q^{\prime}]>0$ it holds that

[TABLE]

Proof.

Let $Q$ be any query sequence such that $\Pr[\operatorname{\text{\sf D}}=Q]>0$ and $\Pr[\mathsf{RAM}(\operatorname{\text{\sf D}})=\mathcal{T}\mid\operatorname{\text{\sf D}}=Q]>0$. Assume by contradiction that $\Pr[\operatorname{\text{\sf D}}=Q^{\prime}]>0$ but $\Pr[\operatorname{\text{\sf D}}=Q^{\prime}\mid\mathsf{RAM}(\operatorname{\text{\sf D}})=\mathcal{T}]=0$ for some query sequence $Q^{\prime}$. By Bayes’ Law,

[TABLE]

which implies that $\Pr[\mathsf{RAM}(\operatorname{\text{\sf D}})=\mathcal{T}\mid\operatorname{\text{\sf D}}=Q^{\prime}]=0$. On the other hand, by $\epsilon$-differential privacy, we have that for all $q$ such that $\Pr[\operatorname{\text{\sf D}}=Q]>0$,

[TABLE]

providing our contradiction. ∎

We are now ready to prove our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ lower bound. For the sake of the completeness, we suppose that the ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme only retrieves the desired block with probability at least $1-\alpha$ based only on internal randomness. We show that:

Theorem 3.7.

If $\mathsf{RAM}$ is an ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ scheme in the balls and bins model with error probability $\alpha\geq 0$ then $\mathsf{RAM}$ performs

[TABLE]

expected amortized operations per query when the client has storage for $c$ balls.

Proof.

Fix $Q=(q_{1},\ldots,q_{l})$ to be any sequence of queries of length $l>c$. Then, ${\sf{client}}_{c}$ can only perform upload/download operations with ${\sf{server}}_{M}$. Denote the expected amortized bandwidth by $k$. The server locations of all operations are given by the transcript, $\mathsf{RAM}(q)$. However, which of the $c$ client locations used for each operation remains hidden. For each download and upload, there are $c$ possible execution paths, one path for each of the $c$ client memory locations. Also, after each operation, ${\sf{client}}_{c}$ may use the data stored in any of the $c$ client memory locations to answer a query. Altogether, at most $kl$ blocks of bandwidth are used for all $l$ queries. Therefore, there are at most $c^{2kl}$ different sequences of blocks that may be returned. Since $\mathsf{RAM}$ can only fail with $\alpha$ probability for each query, it must satisfy at least $(1-\alpha)^{l}n^{l}$ different access patterns. So, $c^{2kl}\geq(1-\alpha)^{l}n^{l}$.

Let $\mathcal{T}$ be any transcript with positive probability and consider the uniform distribution $U$ over the set $[n]^{l}$ of query sequences of length $l$. By Lemma 3.6, there exists $Q\in[n]^{l}$ such that

[TABLE]

For every $Q^{\prime}\neq Q$, by Lemma 3.5, we have

[TABLE]

Finally, we know that

[TABLE]

completing the proof. ∎

Unlike ${\mathsf{DP\text{-}\mathsf{IR}}}$, we do not need a non-zero error probability to get a constant overhead scheme. From the above lower bound, it seems like the best one can achieve is a perfectly correct ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ with $\epsilon=\Theta(\log n)$ that only requires $O(1)$ overhead with small client storage. In Section 6, we show that such a scheme does exist.

Discussion about lower bounds.

It has been shown that previous ORAM lower bounds come with many cavaets. The first lower bound (GO96, ) in the balls and bins model was for statistical security as pointed out in (BN16, ). The assumption of statistical security is troublesome due to the fact that all ORAM schemes require the use of encryption which only provides computational security. To abstract away this issue, it is assumed that records were opaque balls and were hidden even against computationally unbounded adversaries. However, this abstraction was still not sufficient since the majority of ORAM schemes still required the use of pseudorandom functions. Furthermore, the ${\mathsf{DP\text{-}\mathsf{RAM}}}$ constructions of (WCM16, ) and (CCM18, ) require encryption and are burdened by the same cavaets. We explain why these issues do not apply to our results. In our work, we consider typical differential privacy notions (not the computational variants described in (MPR09, )) where adversaries are computationally unbounded. Our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme in Section 6 when only allowing retrievals does not require encryption and provides differentially private access to public data against computationally unbounded adversaries. All ORAM schemes still require encryption even when only retrievals are permitted. In the case that we wish to store encrypted data or protect overwrites in our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme, we must apply the abstraction of opaque balls to go around encryption. With this abstraction, our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme is differentially private. Recent results in (LN18, ) present $\Omega(\log(n/c))$ lower bounds for ORAM against computational adversaries with passive servers and general storage schemes. Furthermore, work in (PY18, ) extends the lower bound for ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{RAM}}}}}$ where $\epsilon=O(1)$ and $\delta\leq 1/3$. However, the lower bounds in (PY18, ) have an exponentially worse dependency on $\epsilon$ compared to our results.

4. An Insecure Construction

Before presenting our constructions, we consider a simple and tempting, but insecure, construction. The lower bounds presented in Section 3 show that the best privacy any constant overhead storage primitive can achieve will be $\epsilon=\Theta(\log n)$. With such weak privacy requirements, it might seem very easy to construct these primitives at first. We caution that schemes with these weak privacy requirements must be constructed carefully as slight variants of our later schemes could also end up being insecure. To our knowledge, our schemes are the simplest constructions that achieve $\epsilon=\Theta(\log n)$ differential privacy and small overhead.

The main idea of the strawman solution derives from the fact that $\epsilon=\Theta(\log n)$. As a result, the desired block should be queried with probability a multiplicative factor of $\mathsf{poly}(n)$ larger to compared to any other block. To achieve this, one could query the desired block with probability $1$ and all other blocks with probability $1/n$. This scheme would have $O(1)$ bandwidth in expectation, perfect correctness and no client storage requirements. However, we show that this scheme is really an ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ with $\epsilon=\Theta(\log n)$ and $\delta=(n-1)/n$. Denote the above scheme by $\mathsf{IR}$ and $\mathsf{IR}(i)$ the set of blocks returned when querying for $i$. Pick any two queries $i\neq j$. Note, $\Pr[B_{i}\notin\mathsf{IR}(i)]=0$ and $\Pr[B_{i}\notin\mathsf{IR}(j)]=(n-1)/n$. Then, $(n-1)/n=\Pr[B_{i}\notin\mathsf{IR}(j)]\leq e^{\epsilon}\Pr[B_{i}\notin\mathsf{IR}(i)]+\delta$ which means that $\delta\geq(n-1)/n$. Therefore, the above scheme would not provide as much privacy as possible as $\delta$ approaches $1$ as $n$ increases.

We use the above strawman to show that attentiveness and rigor are required when constructing differentially private storage schemes even with such weak privacy guarantees. The schemes that we will present for ${\mathsf{DP\text{-}\mathsf{IR}}}$, ${\mathsf{DP\text{-}\mathsf{RAM}}}$ and ${\mathsf{DP\text{-}KVS}}$ are the simplest algorithms that, to our knowledge, achieve our desired privacy of $\epsilon=\Theta(\log n)$ and small overhead in efficiency.

5. DP-IR Construction

For errorless ${\mathsf{DP\text{-}\mathsf{IR}}}$, the asymptotically optimal balls and bins algorithms is required to download the entire database regardless of the privacy budget. With active servers that can perform computation, servers must perform $n$ operations which is identical to PIR. Therefore, there is no reason to use differentially private access when oblivious access has the same efficiency.

We now move to the more interesting case of ${\mathsf{DP\text{-}\mathsf{IR}}}$ with errors. By introducing a small amount of error $\alpha>0$, we hope to find algorithms more efficient than PIR for larger privacy budgets where our lower bounds from Section 3 no longer hold. We show that the simplest algorithm ends up being optimal. In particular, the client will download the desired block as well as several other blocks simultaneously. The hope is that the adversary cannot determine the real retrieval from all the fake retrievals. In addition, with probability $\alpha$, we only perform fake retrievals and error. The pseudocode of this scheme can be found in Appendix G. The proof of the following theorem is found in Appendix B

Theorem 5.1.

For any $\epsilon\geq 0$, there exists a ${\epsilon\mathsf{\text{-}{\mathsf{DP\text{-}\mathsf{IR}}}}}$ that returns $O(n/e^{\epsilon})$ blocks for any constant error probability $\alpha>0$.

The above upper bound asymptotically matches the lower bound of Theorem 3.4 for all values of $\epsilon\geq 0$. Furthemore, by fixing the privacy budget to be $\epsilon=\Theta(\log n)$, we can achieve a constant overhead ${\mathsf{DP\text{-}\mathsf{IR}}}$ scheme with the best privacy according to Theorem 3.4.

6. DP-RAM Construction

In this section, we give an errorless construction ${\mathsf{DP\text{-}\mathsf{RAM}}}$ supporting both retrieval and overwrite operations. Before describing our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme, we note that one could also use the ${\mathsf{DP\text{-}\mathsf{IR}}}$ scheme from Section 5 as a ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme without any client storage requirements. However, this ${\mathsf{DP\text{-}\mathsf{IR}}}$ scheme has non-zero error probabilities which is inherently unavoidable for ${\mathsf{DP\text{-}\mathsf{IR}}}$ due to our lower bounds in Section 3. On the other hand, our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ lower bounds do not preclude the existence of a perfectly correct ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme with $\epsilon=\Theta(\log n)$. In this section, we work towards constructing such a ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme.

Our scheme will require the client to store some records in a local stash. Our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme is parameterized by a probability $p$ describing the independent probability that each record is stored in the stash. We assume that $(\operatorname{\text{\sf Enc}},\operatorname{\text{\sf Dec}})$ is an IND-CPA symmetric-key encryption scheme. The server’s storage will consist of an array, $A$, of $n$ records.

The setup phase of ${\mathsf{DP\text{-}\mathsf{RAM}}}$ will consist of populating the server-stored array $A$. For security parameter $\lambda$, key $K\rightarrow\{0,1\}^{\lambda}$ is randomly selected. $A$ is initialized by setting $A[i]=\operatorname{\text{\sf Enc}}(K,B_{i})$ for $i\in[n]$. The stash is initialized by independently selecting each record to be in the stash with probability $p$. In addition, the client keeps the key $K$ in local storage.

The querying (either retrieval or overwrite) for a record $B_{i}$ consists of two phases: the download phase followed by the overwrite phase. In the download phase, the client looks for $B_{i}$ in the stash. If $B_{i}$ is found, then $B_{i}$ is removed and returned. The client asks the server for $A[j]$, with $j$ chosen uniformly at random from $[n]$. If, instead, $B_{i}$ is not in the stash, the client asks the server for $A[i]$ that contains an encryption of $B_{i}$. If the client is performing a write operation, then $B_{i}$ is updated with the new version. At this point, the client holds the current version of $B_{i}$.

In the overwrite phase, the current version of $B_{i}$ is added to the stash with probability $p$. If $B_{i}$ is stored in the stash, then another record is randomly selected, downloaded from the server, decrypted and then re-encrypted with fresh randomness and uploaded to the server. If $B_{i}$ is not stored in the stash, then the client asks the server for $A[i]$, discards the record received and then uploads to $A[i]$ a freshly computed ciphertext carrying the current version of $B_{i}$.

The pseudocode of the algorithms are presented in Appendix H. We note that while the above algorithm is simple, the analysis of privacy is quite complicated. We show the following about this scheme when $p\leq\Phi(n)/n$ for any $\Phi(n)=\omega(\log n)$:

Theorem 6.1.

There exists an $O(\log n)\text{-}\mathsf{DP}\text{-}\mathsf{RAM}$ that returns $O(1)$ blocks. For any function $\Phi(n)=\omega(\log n)$, the client stores $\Phi(n)$ blocks of client storage except with probability $\operatorname{\mathtt{negl}}(n)$.

This scheme is, essentially, the best privacy that can be achieved by an errorless ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme with constant overhead according to Theorem 3.7. Since we use encryption, our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ satisfies computational differentially privacy using simulators (see SIM-CDP in (MPR09, )). Our proof uses a simulator that replaces all encryptions of records with randomly generated contents.

Discussion about encryption. The above ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme assumes that both record retrievals and overwrites are permitted. To hide whether queries are overwrites or retrievals, any ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme must use encryption. In the case that we wish to only permit retrievals, we note that the above ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme no longer requires encryption and can provide differentially private access to public data without computational assumptions. In particular, the entire overwrite phase may be skipped. Retrieval-only ${\mathsf{DP\text{-}\mathsf{RAM}}}$ and ${\mathsf{DP\text{-}\mathsf{IR}}}$ only differ by their requirements on the client’s state. Therefore, using client state is another way to bypass the strong lower bound shown in Theorem 3.3 for errorless ${\mathsf{DP\text{-}\mathsf{IR}}}$ schemes.

6.1. Roadmap of the Proof

To start, we show that if we choose that $p\leq c/n$ where $c=\omega(\log n)$, then the client will store at most $O(c)$ blocks. The proof is an application of Chernoff Bounds and postponed to Appendix D.

The technical crux of the privacy analysis lies in bounding the following ratio, for every transcript $\mathcal{T}$ seen by the adversary, and for every two neighboring query sequences $Q$ and $Q^{\prime}$,

[TABLE]

where $\Pr[\mathsf{RAM}(Q)=\mathcal{T}]$ denotes the probability that $\mathsf{RAM}$ executing on $Q$ produces the transcript $\mathcal{T}$. In the first step, we show that the transcript of the adversary at a single query on $B_{q}$ is only dependent on the most recent query that also queries for $B_{q}$. In other words, the ratio is upper bounded by

[TABLE]

where $\mathtt{pr}(Q,i)$ is the most recent query for block $B_{q_{i}}$ before the $i$-th query. The second step consists of giving an upper bound that holds for each factor of the product in the above equation. The third step shows that the upper bound computed in the second step is too pessimistic and that, all factors, except for $3$, in the product in the right hand side of the above equation are $1$ when $Q$ and $Q^{\prime}$ differ in exactly one position.

We now proceed to define notation and terminology used throughout our proof. First, we observe that the transcript of an execution only includes the ciphertexts of the blocks and not the actual content of the blocks transferred. Assuming IND-CPA of the underlying encryption scheme $(\operatorname{\text{\sf Enc}},\operatorname{\text{\sf Dec}})$, it is straightforward to prove that, for a sequence $Q=(q_{1},\ldots,q_{l})$ of $l$ queries, the transcript generated by $Q$ for blocks $B_{1},\ldots,B_{n}$ is indistinguishable from the transcript generated by the same sequence $Q$ for $n$ blocks that are ${\mathbf{0}}$. For this reason, we shall not consider the ciphertexts of the blocks as part of the transcript and consider a transcript $\mathcal{T}$ for query sequence $Q$ as a sequence $\mathcal{T}=((d_{1},o_{1}),\ldots,(d_{l},o_{l}))$ of $l$ pairs $\mathcal{T}_{j}=(o_{j},d_{j})$ of indices of the blocks that are accessed during the download phase and the overwrite phase. We set $\mathcal{T}_{[j]}=((d_{1},o_{1}),\ldots,(d_{j},o_{j})).$ We define $\mathsf{RAM}^{D}_{j}(Q)$ and $\mathsf{RAM}^{O}_{j}(Q)$ to be the random variables of the indices of the download and of the overwrite block of the $j$-th query for all $j\in[l]$. Also, for $S\subseteq[l]$ we define $\mathsf{RAM}_{S}(Q)$ to be the set of random variables $\{\mathsf{RAM}^{D}_{j}(Q),\mathsf{RAM}^{O}_{j}(Q)\}_{j\in S}$. Also, we set $\mathsf{RAM}(Q):=\mathsf{RAM}_{[n]}(Q)$. It turns out to be convenient to extend the random variable $\mathsf{RAM}$ for indices $\pm\infty$ by setting $\mathsf{RAM}_{\pm\infty}^{D}(Q)$ and $\mathsf{RAM}^{O}_{\pm\infty}(Q)$ to be random variables that give probability $1$ to $\perp$.

We also let $\mathtt{pr}(Q,j)$ be the index of the most recent previous query to block $B_{q_{j}}$ that happened before the $j$-th query of $Q$; that is, $\mathtt{pr}(Q,j)=\max\{i<j:q_{i}=q_{j}\}$. If query $q_{j}$ is the first query of sequence $Q$ that asks for block $B_{q_{j}}$, then $\mathtt{pr}(Q,j)=+\infty$. Similarly, we define $\mathtt{nx}(Q,j)$ to be the index of the nearest next query for block $B_{q_{j}}$; that is, $\mathtt{nx}(Q,j)=\min\{i>j:q_{i}=q_{j}\}$. If the $j$-th query of $Q$ is the last query of $Q$ to ask for block $B_{q_{j}}$, then $\mathtt{nx}(Q,j)=-\infty$.

6.2. Step I: Reducing dependencies

The next lemmas outline the dependencies of the random variables of the download blocks and of the overwrite blocks. We start by showing that the overwrite block of each query is independent of all previous history and only depends on the current query.

Lemma 6.2.

For every query sequence $Q$ of length $l$, for every transcript $\mathcal{T}=((d_{1},o_{1}),\ldots,(d_{l},o_{l}))$ and for every $j\leq l$

[TABLE]

Moreover, if $Q$ and $Q^{\prime}$ are two sequences with $q_{j}=q^{\prime}_{j}$, then the distributions $\mathsf{RAM}^{O}_{j}(Q)$ and $\mathsf{RAM}^{O}_{j}(Q^{\prime})$ coincide.

Proof.

The lemma follows by observing that the distribution of the $j$-th overwrite block depends only on whether $B_{q_{j}}$ is added to the block stash during the $j$-th overwrite phase which, in turn, depends only on $q_{j}$ and the random value of $r$ in the overwrite phase of the $j$-th query. ∎

Lemma 6.3.

*For every query sequence $Q$ of length $l$, for every transcript $\mathcal{T}=((d_{1},o_{1}),\ldots,(d_{l},o_{l}))$ and for every $j\leq l$, *

[TABLE]

Moreover, if $Q$ and $Q^{\prime}$ are two sequences with $q_{j}=q^{\prime}_{j}$ and $\mathtt{pr}(Q,j)=\mathtt{pr}(Q^{\prime},j)$ then for all $d\in[n]$ and for all $o\in[n]\cup\{\perp\}$ such that $\Pr[\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}=o]>0$,

[TABLE]

Proof.

The $j$-th download block of sequence $Q$ depends on whether $B_{q_{j}}$ is in the block stash at the start of the $j$-th download phase. We distinguish two cases.

If $\mathtt{pr}(Q,j)=+\infty$, then block $B_{q_{j}}$ has not been queried in the first $j-1$ queries and its probability of being found in the stash at the start of $j$-th download phase is equal to the probability of being placed in the stash by $\mathsf{RAM}.\!\text{\sf Setup}$ which, obviously, is independent from the previous history.

Suppose instead that $\mathtt{pr}(Q,j)\in[j-1]$. Then the probability that $B_{q_{j}}$ is in the stash at the beginning of the $j$-th query depends only on the random variable, $\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}(Q)$, of the overwrite block of query $\mathtt{pr}(Q,j)$. ∎

The first part of Lemma 6.2 and the first part of Lemma 6.3 imply

[TABLE]

Thus, we can write

[TABLE]

6.3. Step II: Upper bounding factors

In the next two lemmata, we give an upper bound on the contribution of each $j\in[l]$ to the product in the equation above.

Lemma 6.4.

Let $Q$ and $Q^{\prime}$ be two query sequences of length $l$. For every transcript $\mathcal{T}=((d_{1},o_{1}),\ldots,(d_{l},o_{l}))$ and every $j\in[l]$

[TABLE]

Lemma 6.5.

Let $Q$ and $Q^{\prime}$ be two query sequences of length $l$. For every transcript $\mathcal{T}=((d_{1},o_{1}),\ldots,(d_{l},o_{l}))$ and every $j\in[l]$

[TABLE]

Both of these lemmata consider the various cases that can occur. As the proofs are case analysis that do not provide better intuition to the problem, we postpone them to Section D.

6.4. Step III: Identifying the many good cases

The bounds given by Lemma 6.4 and Lemma 6.5 would give an $n^{O(l)}$ upper bound on the ratio in Equation 1. This is a very weak bound as it depends on the length $l$ of the sequences. In this section, we tighten the upper bound to $n^{O(1)}$ which is instrumental to prove that $\mathsf{RAM}$ is private with $\epsilon=O(\log n)$. Specifically, the next lemma gives sufficient conditions under which the ratio is actually $1$ and then we show that, if the two sequences only differ in one position, then there are only three values of $j$ for which the conditions are not satisfied and for those position we use the upper bound of the previous section.

The following lemma follows directly from the second parts of Lemma 6.2 and 6.3.

Lemma 6.6.

*For any two sequences $Q$ and $Q^{\prime}$ of the same length, every transcript $\mathcal{T}=((d_{1},o_{1}),\ldots,(d_{l},o_{l}))$ and every $j\in[l]$ with $\mathtt{pr}(Q,j)=\mathtt{pr}(Q^{\prime},j)$ and $q_{j}=q^{\prime}_{j}$, *

[TABLE]

The lemma above says that the distributions of the transcripts associated with two query sequences $Q$ and $Q^{\prime}$ may differ only at indices $j$ for which $\mathtt{pr}(Q,j)\neq\mathtt{pr}(Q^{\prime},j)$ or $q_{j}\neq q^{\prime}_{j}$. The next lemma identifies the indices $j$ for which this happens when $Q$ and $Q^{\prime}$ differ in exactly one position.

Lemma 6.7.

Let $Q$ and $Q^{\prime}$ be two query sequences of length $l$ differing only at position $k\in[l]$. If $j\notin\{k,\mathtt{nx}(Q,k),\allowbreak\mathtt{nx}(Q^{\prime},k)\}$, then $\mathtt{pr}(Q,j)=\mathtt{pr}(Q,j^{\prime})$.

Proof.

For $j<k$, we have $(q_{1},\ldots,q_{j})=(q^{\prime}_{1},\ldots,q^{\prime}_{j})$ and thus $\mathtt{pr}(Q,j)=\mathtt{pr}(Q^{\prime},j)$. For $j>k$ such that $q_{j}\neq q_{k},q^{\prime}_{k}$ we have $q_{j}=q^{\prime}_{j}$ and $\mathtt{pr}(Q,j)=\mathtt{pr}(Q^{\prime},j)$. Next consider the indices $j_{1}<\ldots<j_{l}$ such that $q_{k}=q_{j_{1}}=\ldots=q_{j_{l}}$. Clearly, if $l>0$ then $j_{1}=\mathtt{nx}(Q,k)$ and, for $i=2,\ldots,l$, we have $q_{j_{i}}=q^{\prime}_{j_{i}}$ and $\mathtt{pr}(Q,j_{i})=\mathtt{pr}(Q^{\prime},j_{i})$. A similar argument applies for the indices $j>k$ such $q^{\prime}_{j}=q^{\prime}_{k}$ thus completing the proof of the theorem. ∎

6.5. Wrapping up the proof

Proof of Theorem 6.1.

Bounds on bandwidth and server storage are obvious and the one on client storage follows from Lemma D.1.

Consider sequences $Q$ and $Q^{\prime}$ of length $l$ that differ only in position $k\in[l]$. Let $\mathcal{T}=((d_{1},o_{1}),\ldots,(d_{l},o_{l}))$ be transcript. By Lemma 6.6 and 6.7, we obtain that the ratio $\frac{\Pr[\mathsf{RAM}(Q^{\prime})=\mathcal{T}]}{\Pr[\mathsf{RAM}(Q)=\mathcal{T}]}$ is

[TABLE]

where $S=\{k,\mathtt{nx}(Q,k),\mathtt{nx}(Q^{\prime},k)\}$. By Lemma 6.4 and 6.5,

[TABLE]

Since, the above holds for any single transcript, we get that the ratio holds for any set of transcripts. This implies that $\epsilon=O(\log n)$. ∎

7. DP-KVS Construction

In this section, we present ${\mathsf{DP\text{-}KVS}}$, our $\mathsf{KVS}$ construction with privacy budget $\epsilon=O(\log n)$ and small overhead. Recall that a $\mathsf{KVS}$ is an extension of $\mathsf{RAM}$ where each of the $n$ blocks is identified by a unique key taken from a possibly large universe of keys $U$. In contrast, a block in $\mathsf{RAM}$ is uniquely identified by an integer in $[n]$. One could use the ${\mathsf{DP\text{-}\mathsf{RAM}}}$ construction from Section 6, which results in server storage on the order of $|U|\gg n$. For efficiency, we would like a ${\mathsf{DP\text{-}KVS}}$ scheme that stores $O(n)$ blocks on the server.

Our approach to constructing a ${\mathsf{DP\text{-}KVS}}$ scheme consists of two steps. First, we show that we can construct a ${\mathsf{DP\text{-}KVS}}$ scheme using a ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme and a mapping scheme which associates keys in the universe $U$ to subsets of server storage. Next, we present an efficient mapping scheme by constructing a non-trivial oblivious variant of two-choice hashing (Mitzenmacher01, ) that uses $O(n)$ server storage and which may be of independent interest. Our oblivious two-choice hashing variant hides the number of real items that are stored in each bin at any point in time. Finally, combined with our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ scheme of Section 6, we present a ${(\epsilon,\delta)\mathsf{\text{-}{\mathsf{DP\text{-}KVS}}}}$ with $\epsilon=O(\log n)$ and $\delta=\operatorname{\mathtt{negl}}(n)$ using only $O(\log\log n)$ overhead. While non-constant, this is exponentially better than any previous oblivious $\mathsf{KVS}$ scheme built from ORAMs. Furthermore, for all practical sizes of $n$, $O(\log\log n)$ is very small.

7.1. Composing Mapping Schemes and DP-RAM

In this section, we present a generic reduction of ${\mathsf{DP\text{-}KVS}}$ to a mapping scheme and our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ from Section 6. First, we define mapping schemes. Afterwards, we present the reduction.

A mapping scheme is defined as the tuple $(\Pi,\mathcal{S})$ where $\Pi$ is the mapping function and $\mathcal{S}$ is the storing algorithm. To store $n$ items each uniquely identified by an key from the universe $U$, a mapping scheme arranges the server storage into $b(n)$ buckets each consisting of at most $s(n)$ blocks. Each bucket is uniquely identified by an index from $[b(n)]$. We note that buckets are not necessarily disjoint and the total number of blocks may be much smaller than $b(n)\cdot s(n)$. In addition, the mapping scheme assumes that the client will hold a mapping stash which will contain at most $c(n)$ blocks except with probability $\operatorname{\mathtt{negl}}(n)$. For each item $u\in U$, the mapping function maps $u$ to a subset of at most $s(n)$ buckets defined by $\Pi(u)\subseteq[b(n)]$. For convenience, we denote $k(n):=\max_{u\in U}|\Pi(u)|$. When inserting a new item with identifier $u$, the storing algorithm $\mathcal{S}$ determines whether $u$ is placed into a bucket of $\Pi(u)$ or the mapping stash according to the sizes of buckets in $\Pi(u)$. We now show how to use mapping schemes to construct a ${\mathsf{DP\text{-}KVS}}$.

Our ${\mathsf{DP\text{-}KVS}}$ scheme works as follows. We construct the server storage into $b(n)$ buckets as described by the mapping scheme. We build a ${\mathsf{DP\text{-}\mathsf{RAM}}}$ to be able to query and update the $b(n)$ buckets. In Appendix E, we show that our ${\mathsf{DP\text{-}\mathsf{RAM}}}$ construction from Section 6 remains secure and efficient when querying possibly overlapping buckets with minor modifications. When querying our ${\mathsf{DP\text{-}KVS}}$ for a key $u\in U$, we perform $k(n)$ ${\mathsf{DP\text{-}\mathsf{RAM}}}$ queries to retrieve the $s(n)$ blocks from each bucket in $\Pi(u)$. If $|\Pi(u)|<k(n)$, we pick random buckets to pad $\Pi(u)$ to size $k(n)$. Now, we are guaranteed that if $u$ exists in ${\mathsf{DP\text{-}KVS}}$, it appears in a bucket of $\Pi(u)$ or in the mapping stash and can be thus returned. To update an existing key, we can simply update either the bucket or the mapping stash that contains the block associated to key $u$. For insertion, we can execute the storing algorithm $\mathcal{S}$ as all the contents of buckets $\Pi(u)$ and the mapping stash are available to the client. $\mathcal{S}$ determines the insertion location for the block associated to key $u$. Finally, we execute $k(n)$ ${\mathsf{DP\text{-}\mathsf{RAM}}}$ updates to all buckets in $\Pi(u)$. Only the bucket containing the block associated with key $u$ is updated. The other buckets will perform fake updates where the contents remain unchanged. For read operations, none of the contents of buckets in $\Pi(u)$ will be changed. We prove the following theorem about our ${\mathsf{DP\text{-}KVS}}$ construction.

Theorem 7.1.

For $n$ blocks, the above $\mathsf{KVS}$ scheme is an ${\epsilon\mathsf{\text{-}DP\text{-}KVS}}$ with $\epsilon=O(k(n)\cdot\log n)$ that returns at most $O(k(n)\cdot s(n))$ blocks. For any function $\Phi(n)=\omega(\log n)$, the client stores $O(s(n)\cdot\Phi(n)+c(n))$ blocks of storage except with probability $\operatorname{\mathtt{negl}}(n)$.

Proof.

The bounds on bandwidth and server storage follow from the mapping scheme properties. The client storage bound follows from Lemma D.1 and the mapping scheme properties. Each query results in at most $2\cdot k(n)$ queries over the repertoire $\Sigma$ of the $n$ buckets. By the composition theorem and the discussion in Appendix E, we obtain that $\epsilon=O(k(n)\cdot\log n)$. ∎

7.2. Oblivious Two-Choice Hashing

Before presenting our new mapping scheme, we revisit the two-choice hashing scheme (Mitzenmacher01, ) (see Section A.1 for more details). This scheme considers $n$ buckets to store up to $n$ keys. The mapping function $\Pi:U\rightarrow[n]$ sets $\Pi(u)$ as $k(n):=2$ independently and uniformly at random chosen buckets. Typically, $\Pi$ is succintly represented using two keys, $\mathtt{key}_{1},\mathtt{key}_{2}$, of a pseudorandom function $F$ and by $\Pi(u):=\{F(\mathtt{key}_{1},u),F(\mathtt{key}_{2},u)\}$. The storing algorithm $\mathcal{S}$ for $u\in U$, checks which of the two buckets in $\Pi(u)$ is less loaded and places $u$ into the less loaded bucket. There are several different proofs that show that the largest bucket will contain at most $s(n):=O(\log\log n)$ items except with probability $\operatorname{\mathtt{negl}}(n)$.

Unfortunately, we are unable to use two-choice hashing directly into our ${\mathsf{DP\text{-}KVS}}$ scheme without incurring into a server storage blockup. The ${\mathsf{DP\text{-}KVS}}$ scheme from Section 7.1 requires that all buckets are the same size for privacy. The naive approach is to simply increase all buckets to the worst case size which results in $O(n\log\log n)$ server storage. Instead, we now present a variant of two-choice hashing which will only use $O(n)$ server storage by arranging buckets to share memory.

Our bucket arrangement is best described as $\Theta(n/\log n)$ identical binary trees, each with $\Theta(\log n)$ leaf nodes and $\Theta(\log\log n)$ depth. Leaf nodes are denoted as height 0 and the height increases going towards the root. This results in a total of $\Theta(n)$ nodes over all binary trees. Each node in the tree will be able to store up to $t=\Theta(1)$ blocks. Furthermore, we pick the binary trees such that there are exactly $n$ leaf nodes overall. Finally, there is a single root node that has $\Theta(n/\log n)$ children corresponding to the roots of the $\Theta(n/\log n)$ binary tree roots. We denote this node as the super root. Unlike all other nodes, the super root is stored on the client. We shall show that the probability that the super root holds more then $\Phi(n)$ blocks for any $\Phi(n)=\omega(\log n)$ is negligible in $n$.

Each of the $n$ buckets is uniquely associated with a leaf node. The memory locations of a bucket consist of all the blocks stored in the nodes on the unique path between the leaf node and the super root. Therefore, a bucket consists of $\Theta(\log\log n)$ server memory locations in addition to the memory locations in the super root. We now describe our new storing algorithm, $\mathcal{S}$, given this bucket arrangement. When inserting $u$, $\mathcal{S}$ places $u$ into the node with minimal height (that is, closest to the leaf nodes) in either of the buckets in $\Pi(u)$ with empty space. Note that $u$ might end up being stored in the super root. If all the nodes of both buckets of $\Pi(u)$ are filled then the mapping scheme fails to store $u$. We show that when inserting any set of at most $n$ keys, if we limit the capacity of the super root to $\Phi(n)$, for some $\Phi(n)=\omega(\log n)$, the above mapping scheme fails with probability $\operatorname{\mathtt{negl}}(n)$. The analysis adapts techniques from (ABK99, ).

Theorem 7.2.

Let $\Phi(n)=\omega(\log n)$. The probability that, when inserting $n$ keys, mapping scheme $\mathcal{S}$ places more than $\Phi(n)$ keys into the super root is $\operatorname{\mathtt{negl}}(n)$.

Proof.

Our key observation is that a block is stored in a level-($i$+1) slot if and only if the two selected buckets are filled up to level $i$. If we denote by $H_{i}$ the number of filled node at level $i$, the probability of selecting a bucket that is filled up to level $i$ is $H_{i}\cdot 2^{i}/n$. For a block to be allocated to a slot at level $i+1$, it must be the case that both selected buckets are filled up to level $i$ which has probability $\left(\frac{H_{i}\cdot 2^{i}}{n}\right)^{2}$. For convenience, we define the sequence $\beta_{i}$ by setting $\beta_{0}=\frac{n}{e\cdot 3^{4}}$ and $\beta_{i+1}=\frac{e}{n}\cdot\beta_{i}^{2}\cdot 2^{2(i+1)}$, which we will use later in the proof. First, we present a useful property about the sequence and then we show that the probability that $H_{i}>\beta_{i}$ is $\operatorname{\mathtt{negl}}(n)$. The proof of Lemma 7.3 can be found in Appendix F.

Lemma 7.3.

For all $i\geq 0$,

[TABLE]

Lemma 7.4.

If $\beta_{i}=\omega(\log n)$ then, $\Pr[H_{i}>\beta_{i}]\leq i/n^{\omega(1)}$.

Proof.

We proceed by induction on $i$. The base case $i=0$ is established by setting $c>3^{4}\cdot e$. We next upper bound the probability that after inserting $n$ blocks there are more than $\beta_{i+1}$ filled nodes at level $i+1$, given that we start with $H_{i}$ nodes filled at level $i$. To do so, we define $X_{j}$ to be that 0/1 random variable that is $1$ iff the $j$-th block ends up at level $i+1$. Clearly, $\Pr[X_{j}=1]=\left(\frac{H_{i}\cdot 2^{i+1}}{n}\right)^{2}$ and, if $H_{i}\leq\beta_{i}$,

[TABLE]

We thus have

[TABLE]

where the last inequality follows by Theorem A.2. By Lemma 7.3, the sequence $\beta_{i}$ decreases with $i$ and thus $\beta_{i}\geq\beta_{i+1}=\omega(\log n)$. The proof is then completed by observing that

[TABLE]

using our hypothesis that $\Pr[H_{i}>\beta_{i}]=i/n^{\omega(1)}$. ∎

We now conclude the proof of Theorem 7.2. Fix a function $\Phi(n)=\omega(\log n)$ and let $i^{\star}$ be the largest index such that $\beta_{i^{\star}}\geq\Phi(n)$. By Lemma 7.3, we obtain that $i^{\star}=\Theta(\log\log n)$. Following a reasoning similar to the one adopted in proof of Lemma 7.4, we can prove that the expected number of filled nodes at level $i^{\star}+1$ is at most $\beta_{i^{\star}+1}/e$, given that no more than $\beta_{i^{\star}}$ nodes are filled at level $i^{\star}$. By Lemma 7.4, the condition holds except with negligible probability. From the definition of $\beta_{i^{\star}+1}$, we obtain that, for some constant $\alpha$ such that $c\geq\alpha\cdot\beta_{i^{\star}+1}$, the probability that more than $c$ nodes are filled at level $i^{\star}+1$ is inverse exponential in $c$ by Chernoff Bounds, and thus $\operatorname{\mathtt{negl}}(n)$. ∎

7.3. Wrapping up the Proof

We complete our ${\mathsf{DP\text{-}KVS}}$ construction by observing that the mapping scheme described above has $k(n)=2$, $s(n)=\Theta(\log\log n)$ and $c(n)=\Phi(n)$ for any $\Phi(n)=\omega(\log n)$.

Theorem 7.5.

The above $\mathsf{KVS}$ scheme is a ${\epsilon\mathsf{\text{-}DP\text{-}KVS}}$ with $\epsilon=O(\log n)$ that returns $O(\log\log n)$ blocks. The server uses $O(n)$ blocks of storage and, for any function $\Phi(n)=\omega(\log n)$, the probabilty that the client stores more than $O(\Phi(n)\cdot\log\log n)$ blocks is $\operatorname{\mathtt{negl}}(n)$.

8. Conclusions

We consider privacy-preserving storage protocols with small overhead that could be implemented with large-scale, frequently accessed storage infrastructures without negatively impacting response times or resource costs. Our main question is to find the best privacy that can be achieved by small overhead storage schemes.

We formulate our privacy notion using differentially private access, which is a generalization of the oblivious access provided by both ORAM and PIR. We present strong evidence that the best privacy achievable by any constant overhead differentially private storage schemes must have privacy budgets $\epsilon=\Omega(\log n)$. For ${\mathsf{DP\text{-}\mathsf{RAM}}}$ and ${\mathsf{DP\text{-}\mathsf{IR}}}$, we present constructions with asymptotically optimal $\epsilon=\Theta(\log n)$ privacy budgets and $O(1)$ overhead. For ${\mathsf{DP\text{-}KVS}}$, we present a scheme with asymptotically optimal $\epsilon=\Theta(\log n)$ privacy budgets and only $O(\log\log n)$ overhead which is exponentially better than previous constructions. Our ${\mathsf{DP\text{-}KVS}}$ uses a novel, oblivious variant of two-choice hashing that uses only $O(n)$ server storage that may be of independent interest.

Therefore, we answer that the best privacy achievable by privacy-preserving storage systems with small overhead is differentially private access with $\epsilon=\Theta(\log n)$. On the other hand, any storage scheme achieving stronger privacy most likely must incur non-trivial overhead compared to plaintext access.

Appendix A Tools

In this section we briefly review some tools that we use in the design and in the analysis of our constructions.

A.1. Power of Two Choices

The power of two choices concept was motivated from the classical balls and bins problem. The balls and bins concepts considers $n$ balls and $n$ bins. Each of the $n$ balls chooses a single bin independently and uniformly at random. The load of a bin is the number of balls that occupy the bin. It has been shown that with high probability, the load of every bin does not exceed $O(\log n/\log\log n)$  [21]. Consider the case where each of the $n$ balls now chooses two bins independently and uniformly at random. The ball occupies the least loaded of the two chosen bins. This slight alteration ensures that with high probability, the load of each bin does not exceed $O(\log\log n)$ [41]. This result demonstrates that having two choices significantly improves bounds on the maximum load. Furthermore, it turns out that increasing the number of choices to $d\geq 3$ only improves the maximum load bounds by a constant. It is important to note that the allocation of each is chosen independently from the allocation of all other balls.

Theorem A.1.

At any time, the load of any bin produced by the power of two choices process exceeds $O(\log\log n)$ with probability at most $1/n^{\Omega(\log\log n)}$.

A.2. Chernoff Bound

The next theorem gives a bound on the tails of a binomial distribution that we will use to analyze our constructions. See [42] for a proof.

Theorem A.2.

Let $X_{i}$, for $i=1,\ldots,n$ be independent binary random variables with $\Pr[X_{i}=1]=p$ and let $\mu:=np$. Then for every $t\geq\mu$, it holds that

[TABLE]

and, in particular,

[TABLE]

Appendix B Proof of Theorem 5.1

We remind the reader that, for $i\in[N]$, $\mathsf{IR}(i)$ is the random variable of the set of blocks transferred by the server when the client wishes to access block $B_{i}$. We next compute $\Pr[\mathsf{IR}(i)=\mathcal{T}]$ for a subset $\mathcal{T}$ of $K$ blocks.

Case 1: $B_{i}\in\mathcal{T}$. With probability $1-\alpha$, we know that $B_{i}\in\mathcal{T}$ and the remaining $K-1$ blocks are chosen uniformly at random. On the other hand, with probability $\alpha$, for $B_{i}\in\mathcal{T}$, we need to choose $B_{i}$ as one of the $K$ blocks that are chosen randomly. Therefore,

[TABLE]

Case 2: $B_{i}\notin\mathcal{T}$. In this case, we know with probability $(1-\alpha)$, $\mathcal{T}$ is not possible. With probability $\alpha$, we need to ensure that $B_{i}$ is not one of the $K$ blocks chosen. So,

[TABLE]

Let $Q$ and $Q^{\prime}$ be any two query sequences of length $L$ for which $d_{H}(Q,Q^{\prime})=1$. Since, the above algorithm is stateless, we know that its behavior depends solely on the query index. Therefore, it suffices to consider two different sequences queries of length one, $q\neq q^{\prime}\in[N]$. If we choose a transcript $\mathcal{T}$ such that $B_{q},B_{q^{\prime}}\notin\mathcal{T}$ or $B_{q},B_{q^{\prime}}\in\mathcal{T}$, then we see that $\Pr[\mathsf{IR}(q)=\mathcal{T}]=\Pr[\mathsf{IR}(q^{\prime})=\mathcal{T}]$. If, instead, $B_{q}\in\mathcal{T}$ and $B_{q^{\prime}}\notin\mathcal{T}$ we have

[TABLE]

Since this holds for any single transcript $\mathcal{T}$, the same ratio holds for any set of transcripts completing the proof.

Appendix C Lower Bound for Multiple-Server DP-IR

We extend the single server lower bound to the multiple server IR model with $D$ servers and $D_{A}$ adversarial servers. If $D_{A}=D$, then the scenario collapses to the single adversarial server case of Section 3.2. In this section, we assume that $D_{A}<D$ and there is always at least one honest server. For convenience, we define $t=\frac{D_{A}}{D}$ to be the fraction of servers corrupted by $\mathcal{A}$ where $0<t<1$.

The choice of adversarial servers is modeled as a challenger ${\mathcal{C}}$ with a $\mathsf{IR}$ protocol and an adversary $\mathcal{A}$ with the power to corrupt $t$ fraction of servers. Using the knowledge of $\mathsf{IR}$ but not the internal randomness of $\mathsf{IR}$, $\mathcal{A}$ picks $t$ fraction of the $D$ servers to corrupt. ${\mathcal{C}}$ then runs $\mathsf{IR}$ and $\mathcal{A}$ gets the transcript of downloads sent by $\mathsf{IR}$ to the $t$ fraction of corrupted servers. We denote the adversary’s transcript on query sequence $Q$ as $\mathsf{IR}^{\mathcal{A}}(Q)$. We now proceed to present our lower bound in the multi-server model.

Theorem C.1.

If $\mathsf{IR}$ is a $(\epsilon,\delta)$-DP $D$-server IR in the balls and bins model for any $\epsilon,\delta\geq 0$ and error probability $\alpha<1-(\delta/t)$, then $\mathsf{IR}$ performs

[TABLE]

expected operations.

Proof.

We prove our lower bound for the adversary $\mathcal{A}$ that corrupts $D_{A}\leq tD$ randomly and uniformly chosen servers. We know

[TABLE]

If $B_{i}$ appears in $\mathsf{IR}^{\mathcal{A}}(i)$, $\Pr[B_{i}\in\mathsf{IR}^{\mathcal{A}}(i)]\geq(1-\alpha)t$. The rest of the proof follows identically to the proof of Theorem 3.4. ∎

As a result, we show that the ${\mathsf{DP\text{-}\mathsf{IR}}}$ scheme in [49] is optimal when $t$ is considered constant.

Appendix D Remaining DP-RAM Proofs

In this section we give the proofs that were postponed from Section 6. We start by showing that the client uses $O(c)$ memory except with negligible probability for $c=\omega(\log n)$.

Lemma D.1.

If $p\leq\frac{c}{n}$ and $c=\omega(\log n)$, then $\mathsf{RAM}$ stores at most $O(c)$ blocks in the block stash on client storage at any point in time except with negligible probability.

Proof.

Pick any point in time. We note that each block $B_{i}$ has independent probability $p$ of being stored in the client. We let $X_{i}=1$ if and only if $B_{i}$ is stored on the client memory. Let $X$ be the number of blocks stored in client memory, so $X=X_{1}+\ldots+X_{n}$. So, we know that $\operatorname{\mathbb{E}}[X]=\operatorname{\mathbb{E}}[X_{1}+\ldots+X_{n}]=pn\leq c$. By Chernoff Bounds, for any $\delta>0$,

[TABLE]

which is $\operatorname{\mathtt{negl}}(n)$ when $c=\omega(\log n)$. The lemma follows, by a union bound over all points in time (that is, the length of the queries which is polynomial in $n$). ∎

Proof of Lemma 6.4.

The numerator is trivially upper bounded by $1$ and the denominator is lower bounded by the conjunctive probability. We prove the lemma by showing that the latter is at least $p/n^{2}$. To this aim, we consider three cases depending on the values of $\mathtt{pr}(Q,j)$ and $o_{\mathtt{pr}(Q,j)}$. Note that, if $\mathtt{pr}(Q,j)\neq+\infty$, $q_{\mathtt{pr}(Q,j)}=q_{j}$ by definition.

Case 1: $\mathtt{pr}(Q,j)\neq+\infty$ and $o_{\mathtt{pr}(Q,j)}=q_{j}$.

Consider first the case $d_{j}=q_{j}$. If during the overwrite phase of query $\mathtt{pr}(Q,j)$ block $B_{q_{j}}$ is stored in block stash (and this has probability $p$), then $\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}(Q)=q_{j}$ with probability $1/n$. Then at the download phase of query $j$, $B_{q_{j}}$ is found in block stash and therefore the probability that $\mathsf{RAM}^{D}_{j}(Q)=q_{j}$ is $1/n$. On the other hand if during the overwrite phase of query $\mathtt{pr}(Q,j)$ block $B_{q_{j}}$ is not stored in block stash (this has probability $1-p$), then certainly $\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}(Q)=q_{j}$ and, since $B_{q_{j}}$ is not found in block stash at the start of the download phase of query $j$, $\mathsf{RAM}^{D}_{j}(Q)=q_{j}$ with probability $1$. Altogether, we have $\Pr[\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}(Q)=q_{j}\wedge\mathsf{RAM}^{D}_{j}(Q)=q_{j}]=1-p+1/n^{3}\geq 1/n^{3}.$ If, instead, $d_{j}\neq q_{j}$ then it means that $B_{q_{j}}$ must have been stored in the block stash during the overwrite phase of query $\mathtt{pr}(Q,j)$ (this has probability $p$) and then $\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}(Q)=q_{j}$ with probability $1/n$. Then, since $B_{q_{j}}$ is in stash at the start of the download phase of query $j$ $\mathsf{RAM}^{D}_{j}(Q)=d_{j}$ with probability $1/n$. Altogether, we have $\Pr[\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}(Q)=q_{j}\wedge\mathsf{RAM}^{D}_{j}(Q)=q_{j}]=p/n^{2}.$

Case 2. $\mathtt{pr}(Q,j)\neq+\infty$ and $o_{\mathtt{pr}(Q,j)}\neq q_{j}$.

If $o_{\mathtt{pr}(Q,j)}\neq q_{j}$ then it must be the case that, during the overwrite phase of query $\mathtt{pr}(Q,j)$, block $B_{q_{j}}$ is stored in block stash (and this happens with probability $p$) and $o_{\mathtt{pr}(Q,j)}$ is chosen as overwrite block (and this happens with probability $1/n$). When the download phase of query $j$ starts, block $B_{q_{j}}$ is found in the stash and so the probability that $d_{j}$ is selected as download blocks is $1/n$. Whence $\Pr[\mathsf{RAM}^{D}_{j}(Q)=d_{j}\wedge\mathsf{RAM}^{O}_{\mathtt{pr}(Q,j)}(Q)=o_{\mathtt{pr}(Q,j)}]=p/n^{2}.$

Case 3. $\mathtt{pr}(Q,j)=+\infty$. In this case, $o_{\mathtt{pr}(Q,j)}=\perp$ and $B_{q_{j}}$ has not been queried before query $j$. Therefore, at the start of the download phase of query $j$, $B_{q_{j}}$ is found in block stash with probability $p$. By using arguments similar to the ones of the previous cases, we have

[TABLE]

and, for $d_{j}\neq q_{j}$,

[TABLE]

which completes the proof. ∎

Proof of Lemma 6.5.

As in the proof of the previous lemma, we upper bound the numerator by $1$ and show that the denominator is at least $n/p$ starting from the case $o_{j}=q_{j}$. If, in the overwrite phase of the $j$-th query, block $B_{q_{j}}$ is added to the block stash (and this happens with probability $1/n$) then the same block is chosen as overwrite block with probability $1/n$. In the remaining $1-p$ probability, $q_{j}$ is the overwrite block with probability $1$. Therefore $\Pr[\mathsf{RAM}^{O}_{j}(Q)=q_{j}]=(1-p)+p/n.$ If, instead, $o_{j}\neq q_{i}$, then the only case in which $o_{j}$ is the overwrite block of phase $j$ is when $B_{q_{j}}$ is added to the block stash and this gives $\Pr[\mathsf{RAM}^{O}_{j}(Q)=o_{j}]=p/n,$ for every $o_{j}\neq q_{j}$. For any $j\neq i$, $Q_{j}=Q^{\prime}_{j}$ showing the first point. For any $o$, $\Pr[\mathsf{RAM}^{O}_{i}(Q^{\prime})=o]\geq p/n$ giving the second result. ∎

Appendix E ${\mathsf{DP\text{-}\mathsf{RAM}}}$ Generalization

Our proof of differential privacy of ${\mathsf{DP\text{-}\mathsf{RAM}}}$ of Section 6 can be seen to give a more general form of differential privacy. Suppose ${\mathsf{DP\text{-}\mathsf{RAM}}}$ stores $n$ blocks and there exists a repertoire $\Sigma$ of size $b=O(n)$. We interpret $\Sigma$ as the specification of $b$ buckets where each bucket contains exactly $s$ blocks. Note, buckets may overlap and two different buckets may contain the same block. Our privacy proof of ${\mathsf{DP\text{-}\mathsf{RAM}}}$ carries over to the case in which a query retrieves all blocks in a bucket. In other words, we interpret query sequences $Q$ of length $s\cdot l$ as $l$ subsequences each of length $s$ taken from the $\Sigma$ of size $n$ (that is, each subsequence corresponds to a bucket). In this framework, a query sequence $Q=(q_{1},\ldots,q_{s\cdot l})$ is associated with sequence $Q=(\sigma_{1},\ldots,\sigma_{l})$ of length $l$ over $\Sigma$; that is,

[TABLE]

and $\sigma_{1},\sigma_{2},\ldots,\sigma_{l}\in\Sigma$ and $|\Sigma|=b=O(n)$.

The proof of Section 6 handles the case $s=1$ in which an access sequence is a sequence of $n$ requests taken from the repertoire $\Sigma=[n]$.

To argue the general case let us consider first the case in which the server explicitly stores the subsequences of $\Sigma$; specifically, each element of $\Sigma$ is a sequence of $s$ of the original $n$ blocks and each is stored using storage equal to $s$ blocks. Then an access sequence $Q$ of length $s\cdot l$ over $[n]$ is simply an access sequence $(\sigma_{1},\ldots,\sigma_{l})$ of length $l$ over $\Sigma$ and, as it is easily seen, our proof still works and guarantees differential privacy with $\epsilon=\Theta(\log n)$.

This approach, unfortunately, has the drawback that the server storage grows by a factor of $s$. However, we observe that the blocks that constitute the subsequence of $\Sigma$ need not to be explicitly stored by the server that instead can store just the $n$ original blocks and, each time it receives a request $\sigma\in\Sigma$, the server fetches the needed blocks. Note that this transformation preserves differential privacy with $\epsilon=\Theta(\log n)$ as this property is independent of the actual layout of the parts of the individual atomic blocks (in this case the subsequences $\sigma\in\Sigma$) on which the RAM is built. The only modification required is that when retrieving or updating a bucket $i$, the ${\mathsf{DP\text{-}\mathsf{RAM}}}$ must also check if the any block is stored on the client as part of another bucket $j\neq i$. If so, the block on client storage must be returned as opposed to the one from the server for retrievals. For updates, both the server copy and client copy must be updated.

Appendix F Remaining DP-KVS Proofs

[Proof of Lemma 7.3] We proceed by induction and we verify the base case by plugging in $i=0$. For $i\geq 0$, we have

[TABLE]

Appendix G DP-IR Pseudocode

We present the pseudocode for our ${\mathsf{DP\text{-}\mathsf{IR}}}$ scheme of Section 5.

Appendix H DP-RAM Pseudocode

We present the pseudocode for the algorithm in Section 6.