Fundamental Limits of Coded Caching: From Uncoded Prefetching to Coded Prefetching

TL;DR

This paper establishes a connection between uncoded and coded prefetching schemes in multiuser caching, introducing a general scheme that improves the understanding of the fundamental memory-rate tradeoff.

Contribution

It unifies two major classes of caching schemes, providing a new general scheme that achieves previously unknown tradeoff points.

Findings

A new general caching scheme that bridges uncoded and coded prefetching.

Explicit examples demonstrating improved memory-rate tradeoff points.

A rigorous analysis yielding a new inner bound to the caching tradeoff.

Abstract

In order to characterize the fundamental limit of the tradeoff between the amount of cache memory and the delivery transmission rate of multiuser caching systems, various coding schemes have been proposed in the literature. These schemes can largely be categorized into two classes, namely uncoded prefetching schemes and coded prefetching schemes. While uncoded prefetching schemes in general offer order-wise optimal performance, coded prefetching schemes often have better performance at the low cache memory regime. The significant differences in the coding components between the two classes may leave the impression that they are largely unrelated. In this work, we provide a connection between the uncoded prefetching scheme proposed by Maddah Ali and Niesen (and its improved version by Yu et al) and the coded prefetching scheme proposed by Tian and Chen. A critical observation is first given where a coding component in the Tian-Chen scheme can be replaced by a binary .code, which enables us to view the two schemes as the extremes of a more general scheme. An explicit example is given to show that the intermediate operating points of this general scheme can in fact provide new memory-rate tradeoff points previously not known to be achievable in the literature. This new general coding scheme is then presented and analyzed rigorously, which yields a new inner bound to the memory-rate tradeoff for the caching problem.

Equations

$$(M,R)=\left(\frac{tN}{K},\frac{K-t}{1+t}\right),t=0,1,\dots ,K,$$

$$(M,R)=\left(\frac{tN}{K},\frac{\left(\genfrac{}{}{0ex}{}{K}{t+1}\right)-\left(\genfrac{}{}{0ex}{}{K-\min \{K,N\}}{t+1}\right)}{\left(\genfrac{}{}{0ex}{}{K}{t}\right)}\right),t=0,1,\dots ,K.$$

$$A_{2,3}+A_{1,3}+B_{1,2},A_{2,4}+A_{1,4}+C_{1,2},A_{3,4}+B_{1,4}+C_{1,3},A_{3,4}+B_{2,4}+C_{2,3}.$$

$$(M,R)=\left(\frac{t[(N-1)t+K-N]}{K(K-1)},\frac{N(K-t)}{K}\right),t=0,1,\dots ,K.$$

$$A_{1,2},A_{1,3},A_{1,4},B_{1,2},B_{1,3},B_{1,4},C_{1,2},C_{1,3},C_{1,4}$$

$$A_{3,4},B_{1,2},B_{1,4},B_{2,4},C_{1,2},C_{1,3},C_{2,3},A_{1,3}+A_{2,3},A_{1,4}+A_{2,4},$$

$$(M,R)=\left(\frac{N-1}{K},\frac{N(2K-N)}{2K}\right),$$

$$(M,R)=\left(\frac{N}{Kg},N-\frac{N(N+1)}{K(g+1)}\right),g=1,...,N,$$

$$f(x)=\sum _{i=1}^P{v}_i{x}^{q^{i-1}},v_i\in {\mathbb{F}}_{q^m}$$

$$f(ax+by)=af(x)+bf(y),a,b\in {\mathbb{F}}_q,x,y\in {\mathbb{F}}_{q^m},$$

$$[f({\theta }_1),f({\theta }_2),\dots ,f({\theta }_{P_o})]\cdot G.$$

$$I^{[n]}\triangleq \{k\in I_K:\text{user k requests file }{W}_n\},n=1,2,\dots ,N.$$

$${\oplus }_{k\in \mathcal{B}}{W}_{d_k,\mathcal{B}\setminus k}.$$

$$A_{2,3}+A_{1,3}+B_{1,2},A_{2,4}+A_{1,4}+C_{1,2},B_{1,4}+C_{1,3},B_{2,4}+C_{2,3},A_{3,4}.$$

$$A_{2,3}+A_{1,3}+B_{1,2},A_{2,4}+A_{1,4}+B_{1,2},B_{1,4}+B_{1,3},B_{2,4}+B_{2,3},A_{3,4}.$$

$$A_{2,3}+A_{1,3}+A_{1,2},A_{2,4}+A_{1,4}+C_{1,2},A_{1,4}+C_{1,3},A_{2,4}+C_{2,3},A_{3,4}.$$

$$A_{2,3}+A_{1,3}+A_{1,2},A_{2,4}+A_{1,4}+A_{1,2},A_{1,4}+A_{1,3},A_{2,4}+A_{2,3},A_{3,4}.$$

$${\oplus }_{k\in \mathcal{B}}{W}_{d_k,\mathcal{B}\setminus k}={\oplus }_{\mathcal{P}\in {\mathcal{P}}_{\mathbf{t},\mathbf{d}}}\left[{\oplus }_{n\in \mathcal{P}}\left({\oplus }_{k\in \mathcal{B}\cap I^{[n]}}{W}_{n,\mathcal{B}\setminus k}\right)\right]\Rightarrow {\oplus }_{n\in \mathcal{P}}\left({\oplus }_{k\in \mathcal{B}\cap I^{[n]}}{W}_{n,\mathcal{B}\setminus k}\right),\mathcal{P}\in {\mathcal{P}}_{\mathbf{t},\mathbf{d}},$$

$$\mathbf{d}=(A,A,B,C)=(1,1,2,3),$$

$$R_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\triangleq\sum_{{\boldsymbol{t}}\in\mathcal{T}^{(t)}_{{\boldsymbol{d}}}}\sum_{\mathscr{P}\in\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}}\left[\Bigg{(}\prod_{n\in\mathscr{P}}\binom{m_{n}}{t_{n}}-\prod_{n\in\mathscr{P}}\binom{m_{n}-1}{t_{n}}\Bigg{)}\cdot\prod_{n\in\mathbb{supp}({\boldsymbol{t}})\setminus\mathscr{P}}{m_{n}\choose t_{n}}\right],$$

$$M_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)},k}\triangleq N\left(\genfrac{}{}{0ex}{}{K-1}{t-1}\right)-\Delta M_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)},k},$$

$$\Delta M_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)},k}\triangleq$$

$$\sum _{\begin{array}{c}\mathbf{t}\in {\mathcal{T}}_{\mathbf{d}}^{(t)}:\\ t_{d_k}>0\end{array}}\sum _{\begin{array}{c}\mathcal{P}\in {\mathcal{P}}_{\mathbf{t},\mathbf{d}}:\\ d_k\notin \mathcal{P}\end{array}}\left(\genfrac{}{}{0ex}{}{m_{d_k}-1}{t_{d_k}-1}\right)\left[\left(\prod _{n\in \mathcal{P}}\left(\genfrac{}{}{0ex}{}{m_n}{t_n}\right)-\prod _{n\in \mathcal{P}}\left(\genfrac{}{}{0ex}{}{m_n-1}{t_n}\right)\right)\cdot \prod _{n\in supp(\mathbf{t})\setminus \{\mathcal{P}\cup \{d_k\}\}}\left(\genfrac{}{}{0ex}{}{m_n}{t_n}\right)\right].$$

$$R_{\mathbf{d},{\breve{\mathcal{P}}}_{\mathbf{d}}^{(t)}}\triangleq \min (K-t,\tilde{N})\left(\genfrac{}{}{0ex}{}{K}{t}\right),$$

$$M_{\mathbf{d},{\breve{\mathcal{P}}}_{\mathbf{d}}^{(t)},k}\triangleq N\left(\genfrac{}{}{0ex}{}{K-1}{t-1}\right)-\Delta M_{\mathbf{d},{\breve{\mathcal{P}}}_{\mathbf{d}}^{(t)},k},$$

$$\Delta M_{\mathbf{d},{\breve{\mathcal{P}}}_{\mathbf{d}}^{(t)},k}\triangleq \min (K-t,\tilde{N})\left(\genfrac{}{}{0ex}{}{K-1}{t-1}\right).$$

$$\left(\genfrac{}{}{0ex}{}{a}{b}\right)=\{\begin{array}{ll}0,& \text{if }a<b\\ 1,& \text{if }a\ge 0\text{mbox}andb=0\end{array}.$$

$$\sum _{{\mathcal{P}}_{\mathbf{d}}^{(t)}}{\alpha }_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)}}$$

$${\alpha }_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)}}$$

$$\sum _{{\mathcal{P}}_{\mathbf{d}}^{(t)}}{\alpha }_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)}}{R}_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)}}$$

$$\sum _{{\mathcal{P}}_{\mathbf{d}}^{(t)}}{\alpha }_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)}}{M}_{\mathbf{d},{\mathcal{P}}_{\mathbf{d}}^{(t)},k}$$

$${\mathcal{P}}_{(2,1,0),(1,1,2,3)}=\{\{1,2\}\}=\{\{A,B\}\},$$

Full text

1 Introduction

Caching can be used to relieve contention on communication resources by prefetching data to a local or fast memory space, and thus avoiding data retrieval from the remote or slower data source during peak traffic time. Traditionally, caching has mainly been considered in single user settings, e.g., on-CPU caches vs. RAM in computers, where the hit-ratio is the key measure of performance. As networked systems become more prevalent, caching systems involving multiple users have attracted increasingly more research attention.

In their award-winning article [1], Maddah-Ali and Niesen provided a formal information theoretic formulation for the caching problem in multiuser settings; see Fig. 1. In this formulation, there are $N$ files, each of $F$ bits, and $K$ users. Each user has a local cache memory of capacity $MF$ (thus a normalized capacity of $M$). In the prefetching (or sometimes referred to as the placement) phase, the users can fill their caches with contents from the central server without the knowledge of the precise requests at the deliver phase. In the delivery phase, each user reveals the request for a single file from the central server, and the central server must multicast certain common (and possibly coded) information to all the users in order to accommodate these requests. Since in the prefetching phase, the requests at the later phase are unknown a-prior, the cached contents must be strategically prepared at all the users. The goal is to minimize the amount of multicast information which has rate $RF$ (or equivalently the normalized rate of $R$), under the constraint on the normalized cache memory $M$. There is a natural tradeoff between the amount of cache memory and the delivery transmission rate, which is often referred to simply as the memory-rate tradeoff or the $(M,R)$ tradeoff. It was shown in [1] that in terms of this memory-rate tradeoff, coding can be rather beneficial, while solutions based on uncoded prefetching and delivery will suffer a significant loss. Subsequent works extended it also to decentralized caching placements [2], caching with nonuniform demands [3], online caching placements [4], and hierarchical coded caching [5], and many others.

There were quite a few recent efforts [6, 7, 8, 9, 10, 11, 12, 13] aiming to find better codes with improved memory-rate tradeoff, toward the eventual goal of finding the optimal codes and thus completely characterizing the fundamental limit of this tradeoff. In particular, Yu et al. [11] proposed a strategy that is optimal when prefetching is restricted to be uncoded, which in fact directly improves on the scheme in [1]. The key insight in [11] appears to be that the original delivery strategy in [1] may have redundancy in the transmissions, which can be systematically removed to reduce the delivery rate in some cases. In another recent work, Tian and Chen [10] proposed a coded prefetching and the corresponding delivery strategy, which relies on a combination of rank metric codes and maximum distance separable (MDS) codes in a non-binary finite field. In the regime when the memory size $M$ is relatively small, the scheme in [10] can achieve a better performance than that in [11]. Another code construction using coded prefetching was proposed more recently by Gómez-Vilardebó in [13], which can provide further improvement, over the schemes in [11] and [10], in the low memory regime for a specific range of $(N,K)$. The characterization of the fundamental limit of the memory-rate tradeoff however remains open, which appears to require both improved coding schemes and stronger outer bounding techniques [14, 15, 16, 17].

In this work, we show that the scheme in [10] can be slightly modified, where the MDS code used in the delivery phase can be replaced by a code using only binary additions (XOR). Though the alternative perspective itself does not provide further improvement on the known memory-rate tradeoff, it allows us to make a conceptual connection between the scheme in [10] and that in [11]. It further enables us to view these two schemes as the extremes of a more general scheme. The intermediate operating points of this more general scheme can indeed provide new tradeoff points previously not known in the literature, which we demonstrate using an explicit example for $(N,K)=(3,4)$. Extending this example, a general explicit code construction is then rigorously presented and analyzed, which provides a new inner bound to the fundamental limit of the memory-rate tradeoff region in the caching problem. The inner bound does not have a simple closed form expression, but can be represented as a linear program to facilitate its computation.

The rest of the paper is organized as follows. Section 2 first provides some necessary background on existing results and rank metric codes, and then introduces the notion of transmission type. A critical observation is given in Section 3 which connects the two classes of schemes as extreme cases, and then a new memory-rate pair previous unknown in the literature is produced by considering the intermediate cases for $(N,K)=(3,4)$. The new inner bound is given formally in Section 4, and the corresponding coding scheme, its analysis, the proof of the correctness are given in Section 5. Section 6 concludes the paper with a few remarks. A technical proof is relegated to the appendix.

2 Relevant Results and Preliminaries

In this section, we first briefly review existing results on the coded caching problem, and then provide necessary background on rank metric codes, which serve an instrumental role in the new code construction. A new concept important in our code construction, *i.e., * the transmission type, is then introduced.

2.1 Existing Schemes Using Uncoded Prefetching

The scheme in [1], which uses uncoded prefetching, can achieve the following memory-rate pairs

[TABLE]

and since another trivial point is clearly $(M,R)=(0,N)$, the lower convex hull of them provides an upper bound to the optimal tradeoff, as stated in [1]. More recently, Yu et al. [11] gave a scheme which achieves the memory-rate tradeoff points of

[TABLE]

These points strictly improve the rate component $R$ in (1) when $K-N\geq t+1$. Both schemes in [1] and [11] use the same uncoded prefetching strategy, but the delivery strategy in [11] is a direct improvement to that in [1]. It was shown in [11] that in the restricted class of schemes where only uncoded prefetching is allowed, the tradeoff provided in (2) is in fact optimal. These two coding schemes can roughly be understood as follows.

Choose a fixed integer $t$, where $0\leq t\leq K$, and partition each file into ${K\choose t}$ segments of equal size; each segment is thus uniquely associated with a cardinality-$t$ subset $\mathcal{S}$ of the full user set $\{1,2,\ldots,K\}$, and this segment is placed in the caches of users in $\mathcal{S}$ during the prefetching phase. During the delivery phase, consider each $(t+1)$ subset $\mathcal{B}$ of users: within this group, each user is requesting a segment that is in all the other users’ caches, and the server thus sends the XOR of all such segments of this group. Each user in this group can recover their respectively desired segment, since all other segments involved in this transmission are known to this user. As mentioned earlier, these transmissions as a whole, taken over all the possible choices of $\mathcal{B}$, may in fact have redundancy among themselves (i.e., they are linearly dependent) for certain $(N,K)$ parameters, and eliminating such redundancy results in the scheme in [11].

Let us examine an example with $N=3$ files, denoted as $A,B,C$, respectively, and $K=4$ users. Set the auxiliary variable $t=2$, then each file is partitioned into ${4\choose 2}=6$ segments, for example, file $A$ has segments $A_{1,2},A_{1,3},A_{1,4},A_{2,3},A_{2,4},A_{3,4}$, and the segment $A_{1,2}$ is given to users 1 and 2, etc.. Suppose now the users’ requests are $(A,A,B,C)$, i.e., the first two users request file $A$, the third user requests file $B$, and the fourth user requests file $C$. Consider the set of users $\mathcal{B}=\{1,2,3\}$, associated with which we should transmit $A_{2,3}+A_{1,3}+B_{1,2}$ according to the coding scheme discussed above, where the addition is in the binary field. Clearly the three users in $\mathcal{B}$ can recover their individually desired segments, i.e., $A_{2,3},A_{1,3},B_{1,2}$, respectively. For other subsets of users, the transmissions are formed similarly, and the complete set of delivery transmissions is

[TABLE]

In this particular case, the transmissions of (3) do not have any redundancy.

The schemes in both [8] and [9] use uncoded prefetching, and since the scheme in [11] is optimal for this class of codes, the two schemes in [8] and [9] do not provide any additional improvement over (2).

2.2 Existing Schemes Using Coded Prefetching

Even in the pioneering work [1], it was observed that uncoded prefetching schemes are not sufficient to characterize the fundamental limit of the memory-rate tradeoff, and one code example using coded prefetching was given for the case $(N,K)=(2,2)$ as an illustration. In [6], Chen et al. extended this example to the general case $N\leq K$, and showed that the single memory-rate pair $\left(\frac{1}{K},\frac{N(K-1)}{K}\right)$ is achievable and in fact optimal.

More recently, Tian and Chen [10] proposed a more general scheme with coded prefetching for $N\leq K$. It was shown that the scheme can achieve the memory-rate tradeoff pairs

[TABLE]

With $t=1$, it produces exactly the memory-rate pair given in [6].

The general scheme in [10] is somewhat involved, but the digest is as follows. Each file is again partitioned into ${K\choose t}$ segments of equal size, and given to the relevant users as in [11]; however, instead of directly storing them, each user caches certain linear combinations of these corresponding segments, mixed across all the files. During delivery, each symbol being transmitted is a linear combination of the segments from a single file, that serves two roles: firstly, the segments forming a single linear combination being transmitted are all present at certain user’s cache that is not requesting this file, thus this user can use it to help resolve the cached symbols when sufficient such transmissions are collected; secondly, these segments are not present at some users which are requesting that file, thus can also help them to recover the missing segments. In order to guarantee the decodability, the cached contents and the transmitted contents should be made linearly independent, and for this purpose, rank metric codes can be utilized to produce the cached linear combinations, and MDS codes can be used to produce the delivery transmissions.

Let us consider again the example $(N,K)=(3,4)$ and $t=2$. In this case, the linear combinations of the segments

[TABLE]

are placed at user 1’s cache, where each segment is viewed as a symbol in a large finite field. According to the scheme in [10], there should be a total of 5 linear combinations cached; the coefficients of these linear combinations are not critical in this construction, for which either deterministic rank metric codes can be used, or random assignments can be used with a high probability of being a valid choice in a sufficiently large finite field (which implies the existence of a deterministic assignment). Now consider again the requests $(A,A,B,C)$. In this case, the server will send the following 9 symbols

[TABLE]

where the addition is in the same finite field of the information symbol which is usually not binary. The last two linear combinations can also be viewed as the parity symbols of two MDS codes. Now user 1 collects from (6) the symbols $B_{1,2},B_{1,4},C_{1,2},C_{1,3}$, which, together with 5 cached linear combinations, leads to a total of $9$ linear combinations of the basis in (5). Since the linear combinations are designed to be linearly independent, all of the symbols can be resolved. User 1 then collects $A_{1,3}+A_{2,3},A_{1,4}+A_{2,4}$ from which $A_{2,3}$ and $A_{2,4}$ can be recovered by eliminating $A_{1,3}$ and $A_{1,4}$, since they have been resolved from the cached content. It can be verified in a similar manner that all other users can also recover the requested files, and for any other demand patterns, transmissions of 9 symbols will always suffice. The memory-rate pair achieved by the scheme in [1] is $(M,R)=(\frac{3}{2},\frac{2}{3})$ while the scheme in [10] gives $(M,R)=(\frac{5}{6},\frac{3}{2})$, which are illustrated in Fig. 2.

Amiri and Gunduz [12] showed that the following tradeoff point is achievable when $N\leq K$

[TABLE]

using a coded prefetching scheme. However, it can be verified that the pair $(M,R)$ in (7) is precisely on the time-sharing line between (2) and (4) with $t=1$. More recently, Gómez-Vilardebó [13] showed that the following memory-rate pairs are achievable:

[TABLE]

which can offer further improvement when $N\leq K\leq(N^{2}+1)/2$. The lower convex hull of (2), (4) and (8) provides the best known upper bound to the fundamental limit of $(M,R)$ tradeoff known in the literature.

2.3 Linearized Polynomial and Rank Metric Codes

Similar as in [10], rank metric codes based on linearized polynomials (see [18]) can be used to facilitate our code constructions. The following lemma is relevant in this regard; see, e.g., [19].

Lemma 1**.**

A linearized polynomial in finite field $\mathbb{F}_{q^{m}}$

[TABLE]

can be uniquely identified from evaluations at any $P$ points $x=\theta_{i}\in\mathbb{F}_{q^{m}}$, $i=1,2,\ldots,P$, that are linearly independent over $\mathbb{F}_{q}$.

Another relevant property of linearized polynomials is that they satisfy the following condition

[TABLE]

which is the reason that they are called “linearized”. This property implies the following lemma, the proof of which can be found in [10].

Lemma 2**.**

Let $f(x)$ be a linearized polynomial in $\mathbb{F}_{q^{m}}$ as given in (9), and let $\theta_{i}\in\mathbb{F}_{q^{m}}$, $i=1,2,\ldots,{P_{o}}$, be linearly independent over $\mathbb{F}_{q}$. Let $G$ be a $P_{o}\times P$ full rank (rank $P$) matrix with entries in $\mathbb{F}_{q}$, then $f(x)$ can be uniquely identified from

[TABLE]

With a fixed set of $\theta_{i}\in\mathbb{F}_{q^{m}}$, $i=1,2,\ldots,{P_{o}}$, which are linearly independent, we can view $(v_{1},\ldots,v_{P})$ as information symbols to be encoded, and the evaluations $[f(\theta_{1}),f(\theta_{2}),\ldots,f(\theta_{P_{o}})$] as the coded symbols. This is a $(P_{o},P)$ MDS code in terms of rank metric [18], where $P_{o}\geq P$. More importantly, the above lemma says any full rank (rank $P$) $\mathbb{F}_{q}$ linear combinations of the coded symbols are sufficient to decode all the information symbols. This linear-transformation-invariant property had been utilized previously in other coding problems such as network coding with errors and erasures [20], locally repairable codes with regeneration [21], and layered regenerating codes [22].

The codes thus obtained are not systematic, but they can be converted to systematic codes by viewing the information symbols $(w_{1},w_{2},\ldots,w_{P})$ as the first $P$ evaluations $[f(\theta_{1}),f(\theta_{2}),\ldots,f(\theta_{P})]$, which can be used to find the coefficients of the linearized polynomial $(v_{1},v_{2},\ldots,v_{P})$, and then the additional parity symbols can be generated by evaluating this linearized polynomial at the remaining points $(\theta_{P+1},\ldots,\theta_{P_{o}})$. Systematic rank-metric codes are instrumental in our construction.

2.4 Demand Vectors and Transmission Types

Denote the $N$ files in the system as $W_{1},W_{2},\ldots,W_{N}$, and denote the demands by the users in the delivery phase as ${\boldsymbol{d}}=(d_{1},d_{2},\ldots,d_{K})$, where $d_{k}\in\{1,2,...,N\}$ is the index of the file that user-$k$ requests. For convenience, denote the set $\{1,2,\ldots,n\}$ as $I_{n}$. Recall that once the auxiliary parameter $t$ is fixed, each file $W_{n}$ in the scheme of [1] is the collection of all segments $W_{n,\mathcal{S}}$ where $\mathcal{S}\subseteq I_{K}$ and $|\mathcal{S}|=t$, where $|\mathcal{S}|$ is the cardinality of the set $\mathcal{S}$. For a given demand vector ${\boldsymbol{d}}=(d_{1},d_{2},\ldots,d_{K})$, denote the set of users requesting file $W_{n}$ as

[TABLE]

Further define $m_{n}\triangleq|I^{[n]}|$, $n=1,2,\ldots,N$. In the coding scheme we shall present, an arbitrary element (for example, the minimum element) in $I^{[n]}$ will be chosen, denoted as $\ell^{[n]}$, as the leader of $I^{[n]}$. The support of vector ${\boldsymbol{m}}$ is written as $\mathbb{supp}({\boldsymbol{m}})$, i.e., $\mathbb{supp}({\boldsymbol{m}})=\{n|m_{n}>0\}$, and its cardinality is denoted as $N^{*}=|\mathbb{supp}({\boldsymbol{m}})|$, which is the number of files being requested in ${\boldsymbol{d}}$. For convenience, also define $\tilde{N}\triangleq\min(N,K)$.

The notion of the transmission type is associated with each transmission in the scheme in [1]. For a set of users $\mathcal{B}\subseteq I_{K}$ where $|\mathcal{B}|=t+1$, the associated delivery transmission in the scheme of [1], for a fixed demand vector $(d_{1},d_{2},\ldots,d_{K})$, can be compactly written as the binary field summation

[TABLE]

Each such transmission, or alternatively the subset $\mathcal{B}$, is thus associated with an $N$-dimensional vector ${\boldsymbol{t}}$, whose $n$-th coordinate $t_{n}$ specifies the number of users that are demanding file $W_{n}$ in the set $\mathcal{B}$. We call this vector the transmission type of the subset $\mathcal{B}$. For example, in the $(3,4)$ case discussed above when the demand vector is $(W_{1},W_{2},W_{3},W_{4})=(A,A,B,C)$, the transmission type of the user set $\mathcal{B}=\{1,2,3\}$ is ${\boldsymbol{t}}=(2,1,0)$, and the exact transmission is $A_{2,3}+A_{1,3}+B_{1,2}$ where there are exactly two $W_{1}=A$ symbols involved and one $W_{2}=B$ symbol involved. Similarly, the transmission types of the user sets $\{2,3,4\}$ and $\{1,3,4\}$ are both ${\boldsymbol{t}}=(1,1,1)$.

Denote the collection of all valid transmission types for a given demand vector ${\boldsymbol{d}}$ with the auxiliary parameter being $t$ as $\mathcal{T}^{(t)}_{{\boldsymbol{d}}}$. It is clear that for any valid transmission type ${\boldsymbol{t}}\in\mathcal{T}^{(t)}_{{\boldsymbol{d}}}$, we have $\sum_{n=1}^{N}t_{n}=t+1$, and thus the auxiliary parameter $t$ can be uniquely determined from any valid ${\boldsymbol{t}}$. The support of a transmission type ${\boldsymbol{t}}$ is denoted as $\mathbb{supp}({\boldsymbol{t}})$. With a slight abuse of notation, we write the transmission type of a given set $\mathcal{B}\subseteq I_{K}$, where $|\mathcal{B}|=t+1$, as $\mathcal{T}(\mathcal{B})$.

The notion of transmission type should be contrasted to the notion of demand type introduced in [23], which is a length-$N$ vector formed by sorting $(m_{1},m_{2},\ldots,m_{N})$. This notion is also important in our work, because the symmetry in the proposed code implies that only one demand vector per demand type needs to be considered. Denote the collection of the representative demand vectors, one representative demand vector per demand type, as $\mathcal{D}$.

3 A Hidden Connection and Partial Decomposition

The two schemes in [10] and [1] (and its improved counterpart [11]) may seem very different at the first sight: one uses coded prefeteching and the other uncoded, one is non-binary code while the other is binary, and one relies on sophisticated coding techniques such as rank metric codes and the other only relatively simple combinatorics. Nevertheless, a closer look reveals some curious connections between the two schemes. For example, the tradeoff points in (4) lead to the rate values $R=\frac{N(K-t)}{K}$ for $t=0,1,\ldots,K$, which are exactly the same set of $M$ values given in (2). This connection may or may not be a simple coincidence, however we next describe a much less obvious observation which leads to the main result of this paper.

3.1 A Hidden Connection

Consider again the example case for $(N,K)=(3,4)$ and $t=2$. Let us decompose the transmissions in (3) by separating different files in the same linear combination. For example, the linear combination $A_{2,3}+A_{1,3}+B_{1,2}$ is decomposed into a pair of transmissions $(A_{2,3}+A_{1,3},B_{1,2})$. It can be verified that decomposing all the linear combinations in (3) in fact produces exactly the same set of linear combinations in (6), after removing the repeated transmissions. Thus in this example, the delivery transmissions in the scheme [10] can be obtained by fully decomposing the delivery transmissions of the scheme in [1], when the auxiliary parameter $t$ is chosen to be the same in the two schemes.

We note that the addition in (6) is not in a binary field, while the addition in (3) is in the binary field. However, if a binary extension field $\mathbb{F}_{2^{m}}$ is used in (6), the delivery can indeed be accomplished using only additions of the information symbols in this binary extension field, i.e., the coefficients of the linear combinations are either [math] or $1$. Clearly, such additions are equivalent to additions in the base binary field, when the information and coded symbols are represented in their binary vector form.

For other $(N,K)$ parameters, by replacing the MDS code component in the coding scheme in [10] with such decomposed delivery transmissions from the scheme of [1], an alternative version of the code given in [10] can be obtained. In fact, instead of simply presenting this alternative code construction, an even more general construction shall be provided, based on an example given next.

3.2 Partial Decomposition and a New Code Example

The above observation naturally raises the following question: since the delivery strategy in the scheme of [10] can be viewed as being obtained from fully decomposing the delivery transmissions of the scheme in [1], will partial decomposition, with a correspondingly modified prefetching strategy, produce new memory-rate tradeoff pairs? Next we provide an example code, which shows that the answer to this question is indeed positive.

Consider again the case $(N,K)=(3,4)$ and $t=2$, but this time each user caches 8 (instead of 9 as in Sec. 2.1, or 5 as in Sec. 2.2) linear combinations of the information symbols of the corresponding uncoded file segments. The coefficients of the linear combinations can again be either from deterministic rank metric codes (see Section 5), or generated randomly in a large finite field.

We next argue that delivering a total of $5$ coded symbols is sufficient in this case, which gives an achievable memory-rate pair $(M,R)=(4/3,5/6)$. The memory-rater pair is strictly better than $(4/3,23/27)$ achieved by the lower convex hull of the schemes [11, 10, 13], which is currently the best known upper bound in the literature; see Fig. 2 for an illustration. For completeness, a computer-generated outer bound is also included in the figure, which was obtained in a separate work [14]. Interestingly, both $(M,R)=(3/8,2)$ given by the code in [13] and $(M,R)=(4/3,5/6)$ obtained in this work are in fact on this outer bound, and thus optimal.

Due to the symmetry in the code, we only need to consider the demand vectors $(A,A,B,C)$, $(A,A,B,B)$, $(A,A,A,C)$, and $(A,A,A,A)$.

• •

For the demand $(A,A,B,C)$, instead of fully decomposing the transmissions in (3), we now partially decompose them as

[TABLE]

User 1 first collects $B_{1,4}+C_{1,3}$, and thus together with the 8 cached linear combinations, can resolve all the symbols in (5) since he has a total of $9$ linearly-independent linear combinations of the 9 symbols; now user 1 essentially has uncoded cache contents, and thus can recover the needed file segments of $A$ (which are $A_{2,3},A_{2,4},A_{3,4}$) using the remaining transmissions. Users 2, 3, and 4 can use a similar strategy.

• •

For the demand $(A,A,B,B)$, we can transmit the following symbols

[TABLE]

User 1 can collect $B_{1,4}+B_{1,3}$ in order to resolve the cached symbols, and the decoding is similar to the previous case. It is also obvious user 3 and user 4 can indeed recover file $B$.

• •

For the demand $(A,A,A,C)$, we can transmit

[TABLE]

The decoding strategies of other users are similar to the previous cases, and let us only consider user 3 as an illustration. User 3 can use $A_{3,4}$ to resolve the symbols in the cache, then recover $A_{1,4}$ from $A_{1,4}+C_{1,3}$, $A_{2,4}$ from $A_{2,4}+C_{2,3}$, and $A_{1,2}$ from $A_{2,3}+A_{1,3}+A_{1,2}$.

• •

For the demand $(A,A,A,A)$, we can transmit

[TABLE]

Using a similar strategy as above, it is seen that all users can indeed recover file $A$.

In this example case, the delivery transmissions are obtained by partially decomposing the transmissions in the scheme of [1], and in compensation, the number of cached linear combinations in users’ memory is reduced from that of [1]. The number of linear combinations stored in the cache needs to guarantee that the coded symbols can all be resolved to their uncoded form, after a sufficient number of symbols have been collected from the delivery. The rest of the paper is devoted to the task of using this idea to build a general class of codes which yield a new inner bound to the memory-rate tradeoff.

4 A New Inner Bound to the Optimal Memory-Rate Tradeoff

We first formally define the partial decomposition patterns, and then present the new inner bound. The prefetching strategy and the delivery strategy behind this new bound are presented and analyzed in the next section.

4.1 A Formal Description of Partial Decomposition

Fix the auxiliary parameter $t\in I_{K}$, and for now also consider a fixed demand vector ${\boldsymbol{d}}$. A valid partial decomposition pattern on a transmission type ${\boldsymbol{t}}$ is specified by a partition $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$ on $\mathbb{supp}({\boldsymbol{t}})$, i.e., the elements of $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$ are mutually exclusive and jointly exhaustive subsets of $\mathbb{supp}({\boldsymbol{t}})$. For a given transmission type ${\boldsymbol{t}}$ and its partial decomposition pattern $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$, the decomposed transmissions are formed by keeping the symbols in the same partition in $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$ together, but those across partitions separated. More precisely, let $\mathcal{T}(\mathcal{B})={\boldsymbol{t}}$, then the transmission (13) can be rewritten and thus decomposed as

[TABLE]

where $\mathscr{P}\subseteq\mathbb{supp}({\boldsymbol{t}})$ is used to enumerate over the partitions specified by $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$. Note that $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$ is the decomposition pattern for a transmission type ${\boldsymbol{t}}$ (and a demand vector ${\boldsymbol{d}}$), which implies that the transmissions of the same transmission type are not allowed to use different decomposition patterns. In order to specify the delivery transmissions for a given demand vector ${\boldsymbol{d}}$, the decomposition patterns for all transmission types should be given, which are written as a set ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}\triangleq\{\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}|{\boldsymbol{t}}\in\mathcal{T}^{(t)}_{{\boldsymbol{d}}}\}$.

Consider again the example for $(N,K)=(3,4)$: suppose the demand vector is

[TABLE]

and for the transmission type ${\boldsymbol{t}}=(1,1,1)$, the decomposition pattern is $\mathcal{P}_{(1,1,1),(1,1,2,3)}=\{\{1\},\{2,3\}\}.$ With these settings the two transmissions $A_{3,4}+B_{1,4}+C_{1,3}$ and $A_{3,4}+B_{2,4}+C_{2,3}$ in the coding scheme [1] will be decomposed into $\{A_{3,4},B_{1,4}+C_{1,3}\}$ and $\{A_{3,4},B_{2,4}+C_{2,3}\}$, respectively.

For any demand vector ${\boldsymbol{d}}$, a special uncoded transmission pattern, denoted as $\breve{{\boldsymbol{\mathcal{P}}}}_{{\boldsymbol{d}}}^{(t)}$, is also allowed. When $K-t\geq\tilde{N}$, this strategy corresponds to directly transmitting a subset of files in the uncoded form. For general parameters, the transmission strategy will be given more precisely in Section 5.2. The introduction of this pattern is motivated by the coding strategy in [10] when $N^{*}<\tilde{N}$.

4.2 A New Inner Bound

Define the following quantity for any transmission pattern ${\boldsymbol{\mathcal{P}}}_{{\boldsymbol{d}}}^{(t)}$ except ${\boldsymbol{\mathcal{P}}}_{{\boldsymbol{d}}}^{(t)}=\breve{{\boldsymbol{\mathcal{P}}}}_{{\boldsymbol{d}}}^{(t)}$

[TABLE]

and for $k=1,2,\ldots,K$,

[TABLE]

where

[TABLE]

For the special transmission pattern ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}=\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}}$, the corresponding quantities are defined as

[TABLE]

and for $k=1,2,\ldots,K$,

[TABLE]

where

[TABLE]

In the above, the following convention for the degenerate cases of combinatorics has been used

[TABLE]

Intuitively speaking, the vector $(M_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}},1},...,M_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}},K},R_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}})$ provides the cache memory requirements at the users and the rate requirement on the delivery transmission in the proposed coding scheme, when the demand vector ${\boldsymbol{d}}$ and the decomposition patterns ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}$ are fixed. Thus these numbers roughly provide the memory-rate tradeoff for the specific demand vector for a fixed decomposition pattern.

We first observe that there may still be unbalance among the cache memory requirements at different users $(M_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}},1},...,M_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}},K})$, meaning that different users may have different cache memory requirements under the decomposition pattern ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}$. This issue can be mitigated by coding across multiple instances in the prefetching phase and then producing the delivery transmissions using multiple different decomposition patterns on different instances to achieve better balance among users; note that simple space-sharing is not sufficient to achieve such a performance. A second important observation is that regardless the demand vectors or the decomposition patterns, the caching strategy can essentially be kept the same, which is to store a certain number of linear combinations of a fixed set of symbols. The two observations lead to the following definition and the main theorem below, the formal proof of which will be given in the sequel.

Define the region $\mathcal{R}^{(t)}$ to be the collection of the memory-rate pairs $(M,R)$ such that there exists a set of real-valued $\{\alpha_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$ such that

[TABLE]

The auxiliary variables $\{\alpha_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$ serve a similar role to the time-sharing variables, however the region cannot be directly obtained by the time-sharing argument, and is instead obtained by a slightly more elaborate coding approach. We need the following technical definition 111Here we allow a sequence of codes to achieve the $(M,R)$ pair in an asymptotic manner, i.e., approaches this normalized memory-rate pair as the size of the file grows to infinity. Strictly speaking, this approach of definition is not necessary for us, since the quantities we obtain in (19) and (20) are always integers, and thus the extreme points of the constrained polytope in (26)-(30) will always be rational, which can be achieved precisely using the proposed scheme. Then a time-sharing argument can be invoked to argue any irrational-valued memory-rate pairs in the region can be achieved. Definition 1 however avoids this line of argument altogether. to state the main result, which is a new inner bound to the memory-rate tradeoff region.

Definition 1**.**

A memory-rate pair $(M,R)$ is called achievable, if for any $\delta>0$, and for any sufficiently large file size $F$, there exists a code with a normalized memory size no greater than $M+\delta$ and a normalized transmission rate no greater than $R+\delta$.

Theorem 1**.**

For any $t=0,1,2,\ldots,K$, any $(M,R)\in\mathcal{R}^{(t)}$ is achievable. As a consequence, the convex closure $\mathbf{cl}\left(\cup_{t=0,\ldots,K}\mathcal{R}^{(t)}\right)$ is achievable, where $\mathbf{cl}(\cdot)$ means the convex closure.

The proof of this theorem will be given in Section 5. We also have the following corollary, whose proof is given in the appendix.

Corollary 1**.**

The memory-rate pairs in (2) and those in (4) are in the region $\mathbf{cl}\left(\cup_{t=0,\ldots,K}\mathcal{R}^{(t)}\right)$.

Since $\mathcal{R}^{(t)}$ is a polytope constrained by the conditions in (26)-(30), $\mathbf{cl}\left(\cup_{t=0,\ldots,K}\mathcal{R}^{(t)}\right)$ is also a polytope. Using standard technique [24], $\mathbf{cl}\left(\cup_{t=0,\ldots,K}\mathcal{R}^{(t)}\right)$ can be conveniently written as a region constrained by only linear constraints, and thus its boundary can be efficiently computed using linear programming.

To illustrate Theorem 1, we show that for the case $(N,K)=(3,4)$, the aforementioned new memory-rate pair $(\frac{4}{3},\frac{5}{6})$ is indeed in the region $\mathcal{R}^{(2)}$. For this purpose, we need to find a set of $\{\alpha_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$ such that the conditions in (26)-(30) hold for each ${\boldsymbol{d}}\in\mathcal{D}$.

• •

For ${\boldsymbol{d}}=(A,A,B,C)=(1,1,2,3)$, let $\alpha=1$ for the decomposition pattern ${\boldsymbol{\mathcal{P}}}^{(2)}_{(1,1,2,3)}$

[TABLE]

which is exactly the decomposition pattern used for (14). It can be verified that here

[TABLE]

using (19), and

[TABLE]

where the only nonzero term comes from the transmission type $(1,1,1)$ and partition $\mathscr{P}=\{2,3\}=\{B,C\}$ in (21). It can be verified similarly that $M_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}},k}=8$ for $k=2,3,4$.

• •

For ${\boldsymbol{d}}=(A,A,B,B)=(1,1,2,2)$, two decomposition patterns are used: the first is the one without any decomposition, and the second is

[TABLE]

Note that this suggests a different coding approach than that used in the example of Section 3: the existence of two decomposition patterns implies that we can achieve this memory-rate pair by coding across two instances, using the two decomposition patterns given above. It is clear that for the first pattern

[TABLE]

and it can be verified that for the second pattern

[TABLE]

It is clear that choosing $\alpha=0.5$ for both patterns satisfies the conditions (26)-(30).

• •

For ${\boldsymbol{d}}=(A,A,A,C)=(1,1,1,3)$, again two decomposition patterns are used: the first is the one without any decomposition, and the other is

[TABLE]

This case is similar to the previous one, and the parameter $\alpha$ can also be chosen to be $0.5$ each.

• •

For ${\boldsymbol{d}}=(A,A,A,A)=(1,1,1,1)$, two decomposition patterns are used: the first is the one without any decomposition, and the second is the special uncoded transmission. For the first pattern

[TABLE]

and for the second pattern

[TABLE]

We can choose $\alpha=\frac{5}{6}$ for the first pattern and the conditions (26)-(30) indeed hold.

5 The New Coding Scheme

We first give the prefetching strategy and the delivery strategy. The correctness of the code is then proved, which establishes Theorem 1.

5.1 The Prefetching Strategy

The prefetching strategy is in fact rather straightforward, which is to encode the symbols allocated to a user using a rank metric code to produce the linear combinations. However, since we allow coding across multiple instances, a technical issue arises as what are the proportions of different delivery patterns. These values are needed to determine two parameters: the total number of instances to code across, and the total number of coded symbols to cache. To address this technical issue, we consider the following line of argument.

Suppose a memory-rate tradeoff pair $(M,R)\in\mathcal{R}^{(t)}$. The definition of $\mathcal{R}^{(t)}$ implies that there exists a set of $\{\alpha_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$ for which the conditions in (26)-(30) hold. Let us assume that a positive integer $r$ is chosen such that there exists a set of non-negative integers $\{r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$

[TABLE]

Clearly $\epsilon$ can be arbitrarily small by choosing $r$ sufficiently large. Essentially, during the delivery phase, for each demand type ${\boldsymbol{d}}$, within the total of $r$ instances that are being coded across, we will use the decomposition pattern ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}$ on $r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}$ of them during delivery.

Let us now fix $r$ and $\{r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$. For ${\boldsymbol{d}}\in{\mathcal{D}}$, define the memory-rate pair

[TABLE]

Let us also define $M^{\prime}_{r}\triangleq\max_{{\boldsymbol{d}}\in{\mathcal{D}}}M^{\prime}_{{\boldsymbol{d}}}$ and $R^{\prime}_{r}\triangleq\max_{{\boldsymbol{d}}\in{\mathcal{D}}}R^{\prime}_{{\boldsymbol{d}}}$, which will be the effective memory-rate pair of this code.

The key design constraint is that the prefetching strategy needs to be independent of the demand vector, which we describe next. In the proposed code, each file contains $r{K\choose t}$ symbols. Each symbol is thus denoted as $W^{(i)}_{n,\mathcal{S}}$, where $i\in I_{r}$, $n\in I_{N}$ and $\mathcal{S}\subseteq I_{K}$ with $|\mathcal{S}|=t$, is assumed to be a symbol in $\mathbb{F}_{2^{m}}$ for some sufficiently large $m$ to be specified shortly. Each file symbol (segment) will be provided to $t$ users as indicated by $\mathcal{S}$, to be stored as a component of some linear combinations. There are a total of

[TABLE]

symbols allocated to each user, however, only $P_{o}-P$ linear combinations of them are stored in the cache, and the parameter $P_{o}$ is directly related to the normalized memory $M^{\prime}_{r}$ as

[TABLE]

Note that $P_{o}$ is always an integer. A $(P_{o},P)$ systematic rank metric code is then used to encode the $P$ symbols at each user, and the $P_{o}-P$ parities of this code are placed at each user’s cache. For such a rank metric code to exist, $m\geq P_{o}$ suffices.

Our plan next is to show that for each ${\boldsymbol{d}}\in{\mathcal{D}}$, a valid delivery strategy exists with a delivery rate $R^{\prime}_{{\boldsymbol{d}}}$. Then by making the integer $r$ sufficiently large, and choosing the integers $\{r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$ appropriately such that $\epsilon\geq 0$ is made arbitrarily small, we have

[TABLE]

This would prove that the targeted $(M,R)$ is indeed achievable.

5.2 The Delivery Strategy

Consider any demand vector ${\boldsymbol{d}}\in\mathcal{D}$, and recall the parameters $\{r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}}\}$ have been chosen. For convenience, suppose there are a total of $q$ possible decomposition patterns for the demand vector ${\boldsymbol{d}}$, with the first one as ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},1}=\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}}$ which is the special case associated with the uncoded transmissions, and ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},j}$, for $j=2,3,\ldots,q$ other decomposition patterns. For a specific transmission type ${\boldsymbol{t}}$, the corresponding decomposition in ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},j}$ is written as $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}},j}$. Note that $\sum_{j=1}^{q}r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},j}}=r$. For the demand vector ${\boldsymbol{d}}$, the transmissions in the proposed scheme are as given in Algorithm 1.

The transmissions on line-6 and line-13 in Algorithm 1 are uncoded, which stem from the special transmission pattern $\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}}$. We first need to show that the steps on line-4 and line-10 are valid, i.e., such a set $\mathcal{A}^{*}$ or $\mathcal{A}$ can always be found. The latter case is immediate by observing that in this case $K-t\geq N^{*}$, and thus $N^{*}\leq\min(K-t,\tilde{N})\leq\tilde{N}$, and we can always find a set $\mathcal{A}$ such that $\mathbb{supp}({\boldsymbol{m}})\subseteq\mathcal{A}\subseteq I_{N}$. To see that the step on line-4 is also valid, first observe that in this case $K-t\leq N^{*}-1$, and we need to find a set of files $\mathcal{A}^{*}$, such that the given set of users $\mathcal{S}$ (where $|\mathcal{S}|=t$) includes all the users that request files in $\mathcal{A}^{*}$. Suppose we cannot find such a set, this means that there are less than $N^{*}-K+t$ such files, or more than $N^{*}-(N^{*}-K+t)=K-t$ files that are being requests by some users not in $\mathcal{S}$, but this is impossible, since there are only $K-t$ users not in the set $\mathcal{S}$. Thus the supposition is not true, and we can always find such a set $\mathcal{A}^{*}$. It is straightforward to count the total number of transmissions as $r_{{\boldsymbol{d}},\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}}}\min(K-r,\tilde{N}){K\choose t}$, when Algorithm 1 completes line-16.

The transmissions on line-22 in Algorithm 1 have the following property: at least one of the component $W_{n,\mathcal{B}\setminus k}$ (for any fixed superscript (i)) in the transmission must have $k$ that is a leader. It is a simple combinatorial counting task to show that the total number of transmissions in this part of the algorithm is given by $\sum_{j=2}^{q}r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},j}}R_{{\boldsymbol{d}},{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}},j}}$. Essentially we examine all the transmission types, for which the decomposition pattern for the transmission type follows ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},j}$ in the corresponding instances (indexed by the superscript (i)), then count the transmitted linear combinations associated with each partition in this decomposition. We must eliminate the transmissions where no leader in this partition $\mathscr{P}$ is included, which is indeed accounted for as the term $-\prod_{n\in\mathscr{P}}\binom{m_{n}-1}{t_{n}}$ in $R_{{\boldsymbol{d}},{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}},j}}$. Thus after the algorithm runs to completion, a total of $\sum_{j=1}^{q}r_{{\boldsymbol{d}},{\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},j}}R_{{\boldsymbol{d}},{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}},j}}$ symbols are transmitted.

The transmissions on line-22 in Algorithm 1 are a subset of the decomposed transmissions given in (18) for the patterns ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}},j}$, since transmissions without any leader are not allowed as mentioned early. This removes the redundancy in the transmissions after a native decomposition. The precise linear independence relations can be captured in a set of lemmas given in the sequel.

5.3 Three Auxiliary Lemmas

When stating these lemmas, we omit the superscript (i) which is used to index the code instances that are being coded across, as well as the decomposition pattern index $j=2,3,\ldots,q$, since they are irrelevant in these settings. We shall return to this notation later on when it becomes important.

Lemma 3** (Redundancy Reduction Lemma).**

Fix a demand vector ${\boldsymbol{d}}$ and a valid transmission type ${\boldsymbol{t}}$. Designate a subset $\mathscr{P}\subseteq\mathbb{supp}({\boldsymbol{t}})$ as the variable set, and $\bar{\mathscr{P}}=\mathbb{supp}({\boldsymbol{t}})\setminus\mathscr{P}$ as the fixed set. Further fix an arbitrary subset $\mathcal{A}\subseteq\cup_{n\in\bar{\mathscr{P}}}I^{[n]}$ such that $\mathcal{A}\cap I^{[n]}=t_{n}$ for all $n\in\bar{\mathscr{P}}$. Let $\mathcal{L}\triangleq\cup_{n\in\mathscr{P}}\{\ell^{[n]}\}$ be the leader set. Let $\mathcal{Q}_{n}\subseteq I^{[n]}\setminus\ell^{[n]}$ be any subset such that $|\mathcal{Q}|=t_{n}$, and let $\mathcal{Q}\triangleq\cup_{n\in\mathscr{P}}\mathcal{Q}_{n}$. The following equation holds

[TABLE]

Proof.

Observe that

[TABLE]

Consider any fixed $\ell\in\mathcal{L}$, and enumerate all set $\mathcal{V}$ by parts $\mathcal{V}\triangleq(\hat{\mathcal{V}},\tilde{\mathcal{V}})$

[TABLE]

Now consider a fixed $\hat{\mathcal{V}}$, and consider the inner summation

[TABLE]

This is a summation of $(t_{d_{\ell}}+1)t_{d_{\ell}}$ file symbols, each of which is in the form $W_{d_{\ell},(\hat{\mathcal{V}}\cup\mathcal{A})\cup\dot{\mathcal{V}}}$, where $\dot{\mathcal{V}}$ is a subset of $\mathcal{Q}_{d_{\ell}}\cup\{\ell\}$ such that $|\dot{\mathcal{V}}|=t_{d_{\ell}}-1$. Since $|\mathcal{Q}_{d_{\ell}}\cup\{\ell\}|=t_{d_{\ell}}+1$ and the summation form in (47) is symmetric, each file symbol appears exactly twice, which cancel out each other in this binary (extension) field. The proof is thus complete. ∎

The above lemma can be used to show that the decomposed transmissions without any leaders are redundant. To see this, notice that (44) can be rewritten as

[TABLE]

Clearly the summation in the second bracket, which is one of decomposed parts from (18) without any leaders, can be expressed as a linear combination of those in the first bracket, which all have some leaders and are indeed in the delivery transmissions given on line-22 in Algorithm 1. Conversely, the transmissions obtained by directly decomposing those in the delivery transmissions of [1] can be reconstructed using the transmissions given on line-22 in Algorithm 1. Lemma 3 is a generalized version of a similar lemma in [11], which was used to remove the redundancy in the coding scheme given in [1].

The next two lemmas essentially state that there is no further linear redundancy in the transmissions in line-22 of Algorithm 1 to be removed. In order to state the lemmas, the following definition is needed. For any fixed ${\boldsymbol{d}}$, ${\boldsymbol{t}}$, $\mathscr{P}\in\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$, and $\mathcal{A}\subseteq\cup_{n\in\mathbb{supp}({\boldsymbol{t}})\setminus\mathscr{P}}I^{[n]}$ for which $\mathcal{A}\cap I^{[n]}=t_{n}$ for all $n\in\mathbb{supp}({\boldsymbol{t}})\setminus\mathscr{P}$, let

[TABLE]

The next lemma states that the decomposed transmissions can in fact be separated naturally into mutually exclusive groups.

Lemma 4**.**

For any ${\boldsymbol{d}}$ and ${\boldsymbol{\mathcal{P}}}^{(t)}_{{\boldsymbol{d}}}$, and any $({\boldsymbol{t^{\prime}}},\mathscr{P}^{\prime},\mathcal{A}^{\prime})\neq({\boldsymbol{t^{\prime\prime}}},\mathscr{P}^{\prime\prime},\mathcal{A}^{\prime\prime})$, where $\mathscr{P}^{\prime}\in\mathcal{P}_{{\boldsymbol{t^{\prime}}},{\boldsymbol{d}}}$ and $\mathscr{P}^{\prime\prime}\in\mathcal{P}_{{\boldsymbol{t^{\prime\prime}}},{\boldsymbol{d}}}$, we have $\mathcal{W}_{{\boldsymbol{d}},{\boldsymbol{t^{\prime}}},\mathscr{P}^{\prime},\mathcal{A}^{\prime}}\cap\mathcal{W}_{{\boldsymbol{d}},{\boldsymbol{t^{\prime\prime}}},\mathscr{P}^{\prime\prime},\mathcal{A}^{\prime\prime}}=\emptyset$.

Proof.

Suppose that the two sets have a common element $W_{n,\mathcal{C}}$. Then this implies that,

[TABLE]

*i.e., * ${\boldsymbol{t^{\prime}}}={\boldsymbol{t^{\prime\prime}}}$; let us write this transmission type as ${\boldsymbol{t}}$. It also follows that $\mathscr{P}^{\prime}=\mathscr{P}^{\prime\prime}$, since $n\in\mathscr{P}^{\prime}$ and $n\in\mathscr{P}^{\prime\prime}$, but $\mathscr{P}^{\prime}\cap\mathscr{P}^{\prime\prime}=\emptyset$ if $\mathscr{P}^{\prime}$ and $\mathscr{P}^{\prime\prime}$ are distinct; we can thus denote this partition as $\mathscr{P}$. It further follows that $\mathcal{A}^{\prime}=\mathcal{A}^{\prime\prime}=\mathcal{C}\cap\cup_{n\in\mathbb{supp}({\boldsymbol{t}})\setminus\mathscr{P}}I^{[n]}$. This is a contradiction, and thus there is no common element between the two sets. The proof is thus complete. ∎

Lemma 5**.**

Each transmission on line-22 in Algorithm 1 is a linear combination of the elements in a single set $\mathcal{W}_{{\boldsymbol{d}},{\boldsymbol{t}},\mathscr{P},\mathcal{A}}$. All the linear combinations in the transmissions on line-22 of Algorithm 1 using symbols in a single set $\mathcal{W}_{{\boldsymbol{d}},{\boldsymbol{t}},\mathscr{P},\mathcal{A}}$ are linearly independent.

Proof.

The first statement is through direct inspection. We can prove the second statement by analyzing the rank of the corresponding coding matrix, which is however rather long and tedious. We instead prove it through a shortcut, directly utilizing the optimality result established in [11].

Fix a demand vector ${\boldsymbol{d}}\in{\mathcal{D}}$, a transmission type ${\boldsymbol{t}}$, and a partition $\mathscr{P}\in\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}$. We only need to prove that for a fixed $\mathcal{A}$, the transmissions

[TABLE]

when $\mathcal{B}$ ranges over all subsets of $\cup_{n\in\mathscr{P}}I^{[n]}$ that satisfy the condition

[TABLE]

are indeed linearly independent. For this purpose, the exact choice of $\mathcal{A}$ is not relevant, and thus we might as well simply drop it by defining

[TABLE]

which lead to the representation

[TABLE]

where $\mathcal{B}$ has the same range as (52). Now consider a caching system that has files $\{\hat{W}_{n}:n\in\mathscr{P}\}$, the users $\cup_{n\in\mathscr{P}}I^{[n]}$, and the demand vector formed by taking the demand vector ${\boldsymbol{d}}$ at the coordinates $\cup_{n\in\mathscr{P}}I^{[n]}$. The transmissions (54) are in fact part of the transmissions in the scheme in [11] for this system when choosing $t=|\mathscr{P}|-1$. These transmissions cannot possibly be linearly dependent, because if so, the dependence could have been removed to further improve the delivery transmission rate, but it was shown in [11] that this transmission scheme is in fact optimal for each demand vector. The proof is thus complete. ∎

5.4 The Correctness of the Coding Scheme

The next proposition shows that the code is indeed valid for any ${\boldsymbol{d}}\in{\mathcal{D}}$.

Proposition 1**.**

Each user can use the delivery transmissions in Algorithm 1 and the cached content to recover the requested file for any ${\boldsymbol{d}}\in{\mathcal{D}}$.

Proof.

Consider an arbitrary user $k_{o}$, whose demands is $d_{k_{o}}$. From the transmissions in line-1 to line-16, the user can clearly collect for each $\mathcal{S}$, where $k_{o}\in\mathcal{S}$, a total of $r_{{\boldsymbol{d}},\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}}}\min(K-r,\tilde{N})$ uncoded symbols, in the form of $W^{i}_{n,\mathcal{S}}$ for $n\in\mathbb{supp}({\boldsymbol{m}})\setminus\mathcal{A}^{*}$ (or $n\in\mathcal{A}$), and there are clearly ${K-1\choose t-1}$ possible ways to choose such a $\mathcal{S}$. Thus a total of $r_{{\boldsymbol{d}},\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}}}\Delta M_{{\boldsymbol{d}},\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}},k}$ symbols are collected.

Then consider the transmissions on line-22 in the algorithm. For each transmission type ${\boldsymbol{t}}$ such that $d_{k_{o}}\in\mathbb{supp}({\boldsymbol{t}})$, consider every $\mathcal{B}$ such that $\mathcal{T}(\mathcal{B})={\boldsymbol{t}}$, $|\mathcal{B}|=t+1$, $k_{o}\in\mathcal{B}$, and $\left(\bigcup_{n\in\mathscr{P}}\{\ell^{[n]}\}\right)\cap\mathcal{B}\neq\emptyset$. User-$k$ collects the following transmissions in the corresponding instances:

[TABLE]

First note that since $d_{k_{o}}\notin\mathscr{P}$ but $n\in\mathscr{P}$, and $k\in I^{[n]}$, we have $k_{o}\neq k$ in the inner enumeration. Thus $k_{o}\in\mathcal{B}\setminus k$. This implies that all the collected transmissions are linear combinations of the symbols of the form $W^{(i)}_{n,\mathcal{S}}$ where $k_{o}\in\mathcal{S}$, which are the components of the linear combinations stored in the cache of user-$k_{o}$. It is straightforward to count that user-$k$ collects a total of $\Delta M_{{\boldsymbol{d}},{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}},j},k_{o}}$ for the decomposition pattern ${{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}},j}$ such transmissions in (55) for each fixed $i$ value. Together with the cached contents, user-$k_{o}$ has a total of

[TABLE]

linear combinations of the $rN{K-1\choose t-1}$ symbols $W^{(i)}_{n,\mathcal{S}}$ where $k_{o}\in\mathcal{S}$. The collected linear combinations from the delivery transmissions are clearly linearly independent due to Lemma 4 and Lemma 5. Thus these linear combinations that user-$k_{o}$ has gathered can be viewed as a full rank transformation of the corresponding rank metric code symbols, which were produced at the prefetching stage by user-$k_{o}$. By Lemma 2, user $k_{o}$ can recover all these symbols to their uncoded form. At this point, user-$k_{o}$ essentially has all the symbols as if the uncoded prefetching strategy in [1] was used. It remains to argue that user-$k_{o}$ can also recover the file symbols of file $d_{k_{o}}$, which is in the form $W^{(i)}_{d_{k_{o}},\mathcal{S}}$ such that $k_{o}\notin\mathcal{S}$. This is straightforward to check for $i=1,\ldots,r_{{\boldsymbol{d}},\breve{{\boldsymbol{\mathcal{P}}}}^{(t)}_{{\boldsymbol{d}}}}$, because all the needed symbols are transmitted in the uncoded form. For the other cases, since by Lemma 3, the original transmissions in the delivery scheme in [1] can be completely reconstructed using the transmissions in Algorithm 1, and with these transmissions and the uncoded prefetched symbols in the cache, user-$k_{o}$ can indeed recover the missing file symbols through the decoding strategy in [1]. The proof is thus complete. ∎

6 Conclusion

We discovered a connection between the caching strategy in [10] and that in [11], that is decomposing the delivery transmissions in [11] yields those in [10] in some cases. This allows us to view the coding strategy in [10] and that in [11] as the two extremes of a more general scheme. The general scheme can achieve some memory-rate pairs previously unknown in the literature, and can be computed using a linear programming approach.

We note that although the new scheme unifies the codes in [10] and [11], it does not appear to include the codes in [13]. We suspect that an improved code can be found by analyzing the transmission types more carefully to optimize explicitly the decomposition patterns, and then incorporate certain coding components in [13]; this is part of our ongoing work. Our work reported here is information theoretic in nature, and little attention has been paid to the complexity of the codes. Particularly, the proposed code has a large alphabet size, a large subpacketization factor, and needs to code across a large number of instances. Such a code is challenging to use directly in practical systems, and effort toward simplifying it appears worthwhile.

Appendix: Proof of Corollary 1

We first show that (2), i.e., the memory-rate tradeoff points given in [11], can be obtained by specializing (19), (20), and (26)-(30). For this case, the decomposition patterns are given by $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}=\{\mathbb{supp}({\boldsymbol{t}})\}$, i.e., there is no decomposition. As such, (19) reduces to

[TABLE]

The first summation is clearly ${K\choose t+1}$, since it is simply the number of ways to choose $(t+1)$ users from the $K$ users, however counted one transmission type at a time. To simplify the second term in (57), let us consider any demand vector ${\boldsymbol{d}}\in\mathcal{D}$. For any transmission type ${\boldsymbol{t}}$ where there exists $n\in\mathbb{supp}({\boldsymbol{t}})$ such that $t_{n}=m_{n}$, the product $\prod_{n\in\mathbb{supp}({\boldsymbol{t}})}\binom{m_{n}-1}{t_{n}}$ is clearly zero. The second summation can thus be viewed as counting the number of ways to choose $(t+1)$ users, however, with the leaders $\{\ell^{[n]},m_{n}\neq 0\}$ not being chosen; there is clearly ${K-N^{*}\choose t+1}$ ways to do so. This implies that

[TABLE]

Similarly (20) can be simplified for any ${\boldsymbol{d}}\in\mathcal{D}$ as

[TABLE]

since the other term disappears with the choice $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}=\{\mathbb{supp}({\boldsymbol{t}})\}$. The quantities (58) and (59) are independent of ${\boldsymbol{d}}$. It is clear that (2) is identical to (58) and (59) after normalization with the file size $F={K\choose t}$. It is easy to verify that they are indeed in the region $\mathcal{R}^{(t)}$, thus it is clearly inside $\mathbf{cl}\left(\cup_{t=0,\ldots,K}\mathcal{R}^{(t)}\right)$.

Next we show that (4), i.e., the memory-rate tradeoff points given in [10], can also be obtained by specializing (19), (20), and (26)-(30). In this case, the decomposition patterns are given by $\mathcal{P}_{{\boldsymbol{t}},{\boldsymbol{d}}}=\{\{n\}:n\in\mathbb{supp}({\boldsymbol{t}})\}$, i.e., $\mathbb{supp}({\boldsymbol{t}})$ is partitioned into sets, where each set is a singleton. It follows that

[TABLE]

Define $\mathbb{1}_{c}$ to be the indicator function which is equal to $1$ when the condition $c$ holds, and is equal to [math] otherwise. We can now rewrite the summation as

[TABLE]

Notice that the equality

[TABLE]

since the left hand side is the number of ways to choose $t+1$ users among the $K$ users, with $\ell^{[n]}$ already chosen, counted one transmission type at a time. It follows that for any ${\boldsymbol{d}}\in\mathcal{D}$,

[TABLE]

Let us turn to (20), the second term of which in this case can be simplified as

[TABLE]

where the last equality is because for each fixed $n\in\mathbb{supp}({\boldsymbol{m}})\setminus\{d_{k}\}$, the inner summation is simply the number of ways to choose $t+1$ users in the $K$ users, with $\ell^{[d_{k}]}$ and $\ell^{[n]}$ already chosen. Thus we arrive at

[TABLE]

Note that neither (63) nor (65) depends on ${\boldsymbol{d}}$ or $k$.

For $N^{*}=\tilde{N}$, normalizing both of them by ${K\choose t}$ already gives exactly the memory-rate tradeoff pairs in (4). This leaves us only the case when $N^{*}\neq\tilde{N}$ to consider. We shall use two decomposition patterns for this case. Define

[TABLE]

which is clearly non-negative and is associated with the uncoded transmission pattern, and $1-\alpha_{\breve{{\boldsymbol{\mathcal{P}}}}_{{\boldsymbol{d}}}^{(t)}}$ which is also non-negative and is associated with the transmission pattern whose rate and memory are given in (63) and (65). It is easy to check that when $K-t\leq\tilde{N}$,

[TABLE]

and

[TABLE]

On the other hand, when $K-t>\tilde{N}$, (70) still holds, but

[TABLE]

where the last inequality is because

[TABLE]

by the condition $K-t>\tilde{N}$. Thus for $N^{*}\neq\tilde{N}$, we have found the correct $\alpha_{\breve{{\boldsymbol{\mathcal{P}}}}_{{\boldsymbol{d}}}^{(t)}}$ to satisfy the conditions in (26)-(30), and indeed the memory-rate pair (4) is in the region $\mathcal{R}^{(t)}$. The proof is thus complete.