On One Generalization of LRC Codes with Availability

TL;DR

This paper explores a generalized class of locally recoverable codes with intersecting recovery sets, enabling higher code rates and load balancing, supported by theoretical bounds and explicit constructions.

Contribution

It introduces a new generalization of LRC codes with intersecting recovery sets, providing bounds and explicit constructions that improve code rate and load balancing capabilities.

Findings

Derived an upper bound for the code rate.

Provided explicit constructions of the generalized LRC codes.

Showed the benefits of intersecting recovery sets in code performance.

Abstract

We investigate one possible generalization of locally recoverable codes (LRC) with all-symbol locality and availability when recovering sets can intersect in a small number of coordinates. This feature allows us to increase the achievable code rate and still meet load balancing requirements. In this paper we derive an upper bound for the rate of such codes and give explicit constructions of codes with such a property. These constructions utilize LRC codes developed by Wang et al.

Tables

Table 1. TABLE I: Upper bounds on the rate of ( r , t , x ) 𝑟 𝑡 𝑥 (r,t,x) -LRC codes

$(r,t)$	$x=0$	$x=1$	$x=2$	$x=3$
(4, 2)	0.7111	0.7250	0.7429	0.7667
(5, 2)	0.7576	0.7667	0.7778	0.7917
(6, 2)	0.7912	0.7976	0.8052	0.8143
(7, 2)	0.8167	0.8214	0.8269	0.8333
(4, 3)	0.6564	0.6981	0.7516	0.8231
(5, 3)	0.7102	0.7375	0.7708	0.8125
(6, 3)	0.7496	0.7688	0.7915	0.8188
(7, 3)	0.7795	0.7938	0.8103	0.8295

Table 2. TABLE II: Comparison of upper and lower bounds on the rate of ( r , t , x ) 𝑟 𝑡 𝑥 (r,t,x) -LRC codes

$(r,t)$	$x=0$ (WZL)	$R^{\ast }\left(r,t,x=0\right)$	$x=1$	$R^{\ast }\left(r,t,x=1\right)$
(3, 2)	0.6000	0.6429	0.6667	0.6667
(5, 2)	0.7143	0.7576	0.7500	0.7667
(7, 2)	0.7778	0.8167	0.8000	0.8214
(3, 3)	0.5000	0.5786	0.6250	0.6500
(5, 3)	0.6250	0.7102	0.7000	0.7375
(7, 3)	0.7000	0.7795	0.7500	0.7938

Equations

$$\mathcal{C}(i,a{)}_{{\mathcal{R}}_i^j}\cap \mathcal{C}(i,a^{\prime }{)}_{{\mathcal{R}}_i^j}=\varnothing .$$

$$d\le n-k+2-\lceil \frac{t(k-1)+1}{t(r-1)+1}\rceil .$$

$$d\le n-\sum _{i=0}^t\lfloor \frac{k-1}{r^i}\rfloor .$$

$$d\le \underset{\begin{array}{c}1\le x\le \lceil \frac{k-1}{(r-1)t+1}\rceil ;1\le y_j\le t;j\in [x]\\ A<k;x,y_j\in Z^{+}\end{array}}{\min }{d}_{l-opt}^q[n-B,k-A],$$

$$\frac{k}{n}\le R^{\ast }(r,t)=\prod _{i=1}^t\frac{1}{1+\frac{1}{ir}}.$$

$$\mathbf{H}(m,t)=\left(\begin{array}{cc}\mathbf{H}(m-1,t-1)& 0\\ I_{\left(\genfrac{}{}{0ex}{}{m-1}{t-1}\right)}& \mathbf{H}(m-1,t)\end{array}\right),$$

$$|R_i^l\cap R_i^{l^{\prime }}|\le x;$$

$$\mathcal{C}(i,a{)}_{{\mathcal{R}}_i^j}\cap \mathcal{C}(i,a^{\prime }{)}_{{\mathcal{R}}_i^j}=\varnothing .$$

$$\frac{(2r-(s-1)x)}{2}s=\underline{N}(r,j,x)\le \left|\bigcup _{l=1}^j{\mathcal{R}}_i^l\right|\le \overline{N}(r,j,x)=jr.$$

$$\left|\bigcup _{l=1}^t{\mathcal{R}}_i^l\right|$$

$$f(r,t,x)$$

$$|U|\ge nf(r,t,x),$$

$$\tau (v)>\tau (m)\text{for all }m\in {\mathcal{R}}_v^j.$$

$$v_1\to v_2\to ...\to v_l,$$

$$\mathrm{Pr}(v\in U)=\mathrm{Pr}({\cup }_{j=1}^t{A}_j).$$

$$\frac{1}{\overline{N}(r,|S|,x)+1}\le P({\cap }_{j\in S}{A}_j)\le \frac{1}{\underline{N}(r,|S|,x)+1}.$$

$$\mathrm{Pr}({\cup }_{j=1}^t{A}_j)$$

$$E(|U|)$$

$$=\sum _{v\in V}\mathrm{Pr}(v\in U)$$

$$=n\mathrm{Pr}({\cup }_{j=1}^t{A}_j)$$

$$\ge nf(r,t,x).$$

$$R(\mathcal{C})\le R^{\ast }(r,t,x)=1-f(r,t,x).$$

$$k\le n(1-f(r,t,x)).$$

$$\mathbf{H}={\mathbf{H}}_{\text{WZL}}\otimes \underset{x+1}{\underbrace{[11\dots 1]}},$$

$$r$$

$$t$$

$$R$$

$$\mathbf{H}_{\text{WZL}}=\left[\begin{array}[]{cccccccccccc}0&0&0&1&1&1\\ 0&1&1&0&0&1\\ 1&0&1&0&1&0\\ 1&1&0&1&0&0\end{array}\right]$$

$$\mathbf{H}=\left[\begin{array}[]{cccccccccccc}0&0&0&0&0&0&1&1&1&1&1&1\\ 0&0&1&1&1&1&0&0&0&0&1&1\\ 1&1&0&0&1&1&0&0&1&1&0&0\\ 1&1&1&1&0&0&1&1&0&0&0&0\end{array}\right]$$

Full text

I Introduction

A locally recoverable code (LRC) is a code over finite alphabet such that each symbol is a function of small number of other symbols that form a recovering set [1, 2, 3, 4, 5]. These codes are important due to their applications in distributed and cloud storage systems. LRC codes are well-investigated in the literature. The bounds on the rate and minimum code distance are given in [1, 3] for the case of large alphabet size. The alphabet-dependent shortening bound (see [6] for the method explanation) is proposed in [7]. Optimal code constructions are given in [8] based on rank-metric codes (for large alphabet size, which is an exponential function of the code length) and in [9] based on Reed-Solomon codes (for small alphabet, which is a linear function of the code length).

The natural generalization of an LRC code is an LRC code with availability (or multiple disjoint recovering sets). Availability allows us to handle multiple simultaneous requests to erased symbol in parallel. This property is very important for hot data that is simultaneously requested by a large number of users. The case of LRC codes with availability is much less investigated. Bounds on parameters of such codes and constructions are given in [4, 10, 11, 12]. Most of the papers focused on information-symbol locality and availability. In what follows we are interested in all-symbol locality and availability that is preferable in applications as it permits a uniform approach system design.

The property of availability decreases maximum achievable code rate [10]. In this paper we propose a new generalization of LRC codes with availability. Namely, we assume that recovering sets can intersect in a small number of coordinates. This feature allows us to increase the achievable code rate and still meet load balancing requirements.

Our contribution is as follows. We investigate one possible generalization of locally recoverable codes (LRC) with all-symbol locality and availability when recovering sets can intersect in a small number of coordinates. We derive an upper bound for the rate of such codes and give explicit constructions of codes with such a property. These constructions utilize LRC codes developed in [13].

II Preliminaries

II-A LRC codes

Let us denote by $\mathbb{F}_{q}$ a field with $q$ elements. Let $[n]=\{1,2,\ldots,n\}$. The code ${\mathcal{C}}\subset\mathbb{F}_{q}^{n}$ has locality $r$ if every symbol of the codeword $c\in{\mathcal{C}}$ can be recovered from a subset of $r$ other symbols of $c$ [1]. In other words, this means that, given $c\in{\mathcal{C}},i\in[n],$ there exists a subset of coordinates ${{\mathcal{R}}}_{i}\subset[n]\backslash i,|{{\mathcal{R}}}_{i}|\leq r$ such that the restriction of ${\mathcal{C}}$ to the coordinates in ${{\mathcal{R}}}_{i}$ enables one to find the value of $c_{i}.$ The subset ${{\mathcal{R}}}_{i}$ is called a recovering set for the symbol $c_{i}$.

II-B LRC codes with availability

Generalizing this concept, assume that every symbol of the code ${\mathcal{C}}$ can be recovered from $t$ disjoint subsets of symbols of size $r$. More formally, denote by ${\mathcal{C}}_{I}$ the restriction of the code ${\mathcal{C}}$ to a subset of coordinates $I\subset[n]$. Given $a\in\mathbb{F}_{q}$ define the set of codewords ${\mathcal{C}}(i,a)=\{c\in{\mathcal{C}}:c_{i}=a\},\;i\in[n].$

Definition 1

A code ${\mathcal{C}}$ is said to have $t$ disjoint recovering sets if for every $i\in[n]$ there are $t$ pairwise disjoint subsets ${{\mathcal{R}}}_{i}^{1},\dots,{{\mathcal{R}}}_{i}^{t}\subset[n]\backslash i$ such that for all $j=1,\dots,t$ and every pair of symbols $a,a^{\prime}\in\mathbb{F}_{q},a\neq a^{\prime}$

[TABLE]

In what follows we refer these codes as $(r,t)$-LRC codes. We briefly list the existing results below. The first bound for $(r,t)$-LRC codes was given in [14, 15]

[TABLE]

An improvement of this bound was obtained in [10]

[TABLE]

An alphabet-dependent bound was proposed in [12] and has form

[TABLE]

where $A=\sum_{j=1}^{x}(r-1)y_{j}+x$, $B=\sum_{j=1}^{x}ry_{j}+x$ and $d_{l-opt}^{q}$ denote the largest possible minimum distance of a code over $\mathbb{F}_{q}$.

The bound on the rate of $(r,t)$-LRC codes was given in [10]

[TABLE]

This bound was improved in [11] for $t=2$.

In [13] a recursive construction of binary $(r,t)$-LRC codes was proposed. The parameters of these codes are as follows: $n=\binom{r+t}{t}$, $R=\frac{r}{r+t}$ and $d=t+1$. We refer these codes as WZL codes. WZL code is defined by its’ parity-check matrix. Let $m=r+t$. Let us define matrix $\mathbf{H}(m,t)$ as follows. Each row of $\mathbf{H}(m,t)$ is associated with $(t-1)$-subset of $[m]$ sorted in lexicographical order, each column – with $t$-subset of $[m]$ also sorted in lexicographical order. In this case the element $(i,j)$ of $\mathbf{H}(m,t)$ is equal to $1$ if $E_{i}\subseteq F_{j}$, where $E_{i}$ is $(t-1)$-subset of $[m]$ associated with $i$-th row and $F_{j}$ is $t$-subset of $[m]$ associated with $j$-th column. It must be mentioned that $\mathbf{H}(m,t)$ has $\binom{m}{t-1}$ rows and $\binom{m}{t}$ columns and has the following structure:

[TABLE]

where $\mathbf{H}(m,m)=\mathbf{H}(m,1)^{T}=(1,...,1)^{T}$ and $\dim(\mathbf{H}(m,1))=\dim(\mathbf{H}(m,t))=m$.

II-C LRC codes with availability and intersection of recovering sets

Let us give to recovering sets an ability to intersect in at most $x$ positions and define this code as $(r,t,x)$-LRC. More formally, we can say

Definition 2

A code ${\mathcal{C}}$ is said to be $(r,t,x)$-LRC if for every $i\in[n]$ there are $t$ subsets $R_{i}^{1},...,R_{i}^{t}\subset[n]\setminus i$ such, that the following relations follow

for every pair $l,l^{\prime}\in[t]$, $l\neq l^{\prime}$

[TABLE] 2. 2.

for all $j=1,\dots,t$ and every pair of symbols $a,a^{\prime}\in\mathbb{F}_{q},a\neq a^{\prime}$

[TABLE]

In what follows we investigate the parameters of such codes.

III An upper bound on the rate of $(r,t,x)$-LRC codes

III-A The recovery graph

Based on the original idea from [10] we represent locally recoverable codes with locality $r$ and availability $t$ as a graph $G$ in the following way. In accordance to the Definition 2 a coordinate $i$ has $t$ recovering sets ${\mathcal{R}}_{i}^{1},...{\mathcal{R}}_{i}^{t}$, each of size $r$, where ${\mathcal{R}}_{i}^{j}\subset[n]\backslash i$. Define a directed graph $G$ as follows. The set of vertices $V=[n]$ corresponds to the set of $n$ coordinates of the LRC code. The ordered pair of vertices $(i,j)$ forms a directed edge $i\to j$ if $j\in{{\mathcal{R}}}_{i}^{l}$ for some $l\in[t]$. We color the edges of the graph with $t$ distinct colors in order to differentiate between the recovering sets of each coordinate. Note, that as the recovering sets can intersect, then some edges may have several colors. We call $G$ the recovery graph of the code ${\mathcal{C}}.$

In what follows we need the following lemma

Lemma 1

Let $j\in[t]$ and $s=\min\{j,\lfloor r/x\rfloor+1\}$, then

[TABLE]

Proof:

The upper bound is trivial and correspond to the case, when recovering sets do not intersect. To prove the lower bound assume, that any two recovering sets intersect in exactly $x$ positions, we have

[TABLE]

∎

Corollary 1

The out-degree of each vertex $i\in V=V(G)$ is upper bounded with $tr$ and lower bounded with $\underline{N}(r,t,x)$.

III-B Upper bound on the rate

The proof is very similar to the proof from [10]. For the simplicity of the reader we present the proof here in all the details. Let us introduce the following function

[TABLE]

The following lemma will be used in the proof.

Lemma 2

There exists a subset of vertices $U\subseteq V$ of size at least

[TABLE]

such that for any $U^{\prime}\subseteq U$, the induced subgraph $G_{U^{\prime}}$ on the vertices $U^{\prime}$ has at least one vertex $v\in U^{\prime}$ such that its set of outgoing edges $\{(v,j),j\in U^{\prime})\}$ is missing at least one color.

Proof:

For a given permutation $\tau$ of the set of vertices $V=[n]$, we define the coloring of some of the vertices as follows: The color $j\in[t]$ is assigned to the vertex $v$ if

[TABLE]

If this condition is satisfied for several recovering sets ${{\mathcal{R}}}_{v}^{j}$, the vertex $v$ is assigned any of the colors $j$ corresponding to these sets. Finally, if this condition is not satisfied at all, then the vertex $v$ is not colored.

Let $U$ be the set of colored vertices, and consider one of its subsets $U^{\prime}\subseteq U$.

Let $G_{U^{\prime}}$ be the induced subgraph on $U^{\prime}$. We claim that there exists $v\in U^{\prime}$ such that its set of outgoing edges is missing at least one color in $G_{U^{\prime}}$. Assume toward a contradiction that every vertex of $G_{U^{\prime}}$ has outgoing edges of all $t$ colors. Choose a vertex $v\in U^{\prime}$ and construct a walk through the vertices of $G_{U^{\prime}}$ according to the following rule. If the path constructed so far ends at some vertex with color $j,$ choose one of its outgoing edges also colored in $j$ and leave the vertex moving along this edge. By assumption, every vertex has outgoing edges of all $t$ colors, so this process, and hence this path can be extended indefinitely. Since the graph $G_{U^{\prime}}$ is finite, there will be a vertex, call it $v_{1},$ that is encountered twice. The segment of the path that begins at $v_{1}$ and returns to it has the form

[TABLE]

where $v_{1}=v_{l}$. For any $i=1,...,l-1$ the vertex $v_{i}$ and the edge $(v_{i},v_{i+1})$ are colored with the same color. Hence by the definition of the set $U$ we conclude that $\tau(v_{i})>\tau(v_{i+1})$ for all $i=1,\dots,l-1,$ a contradiction.

In order to show that there exists such a set $U$ of large cardinality, we choose the permutation $\tau$ randomly and uniformly among all the $n!$ possibilities and compute the expected cardinality of the set $U.$

Let $A_{v,j}$ be the event that (2) holds for the vertex $v$ and the color $j.$ Since $\Pr(A_{v,j})$ does not depend on $v$, we suppress the subscript $v$, and write

[TABLE]

Let us compute the probability of the event $\cup_{j=1}^{t}A_{j}.$ Note that for any set $S\subseteq[t]$ the probability of the event that all the $A_{j},j\in S$ occur simultaneously can be estimated as follows

[TABLE]

Hence by the inclusion exclusion formula we get

[TABLE]

Now let $X_{v}$ be the indicator random variable for the event that $v\in U$, then

[TABLE]

The proof is completed by observing that there exists at least one choice of $\tau$ for which $|U|\geq\operatorname{\text{\sf E}}(|U|).$ ∎

Theorem 1

The rate of an $(r,t,x)$-LRC code ${\mathcal{C}}$ satisfies

[TABLE]

Proof:

The colored vertices can be viewed as check symbols as they can be recovered from the rest symbols. Thus, the number of information symbols can be estimated as follows

[TABLE]

∎

IV Lower bounds on the rate of $(r,t,x)$-LRC codes

In this section we derive a lower bound on the rate of codes with all symbol locality and availability in which recovering sets can intersect. To find a lower bound we propose the following rather simple code construction. In what follows we explain how to construct a parity-check matrix of a linear $(r,t,x)$-LRC code. We start with a parity-check of $(\tilde{r},\tilde{x})$-WZL code. Let is denote the matrix by $\mathbf{H}_{\text{WZL}}$. The matrix $\mathbf{H}$ of $(r,t,x)$-LRC code is constructed as follows

[TABLE]

where $\otimes$ denotes a Kronecker product of matrices.

As a result, we have a matrix of length $n=(x+1)\binom{\tilde{r}+\tilde{t}}{\tilde{t}}$. It is obvious, that each row of the new matrix will have $(x+1)(\tilde{r}+1)$ ones and the number of positions, in which two recovering sets intersects is equal to $x$. This construction will have the same availability $t$ as it was for the standard WZL code. Thus, the parameters of the resulting code are as follows

[TABLE]

The matrix $\mathbf{H}$ has exactly the same rank as the matrix $\mathbf{H}_{\text{WZL}}$, the rank is equal to $\binom{\tilde{r}+\tilde{t}-1}{\tilde{t}-1}$. Thus, the rate of the resulting code can be calculated as follows

[TABLE]

Example 1

Let us start from an $(2,2)$-WZL code

[TABLE]

and construct a parity-check matrix of an $(5,2,1)$-LRC code. The matrix has a rate $R=0.75$ and shown below

[TABLE]

V Numerical results

In Table I we present the comparison of upper bounds on the rate of $(r,t,x)$-LRC codes for different values of parameter $x$. We see that the value of the upper bound increases with the parameter $x$. In the Table II we present the comparison of the code rate obtained by proposed code construction ($x=1$) and the code rate of WZL codes with the same locality and availability. In addition, we include the values of the upper bounds for the code rate from [10] and the upper bounds for the code rate proposed in this paper. We see, that e.g. for $r=3$, $t=2$ and $x=1$ the lower bound is tight and it is better, then the upper bound for the case of $r=3$, $t=2$ and $x=0$.

VI Conclusion

We investigated one possible generalization of locally recoverable codes (LRC) with all-symbol locality and availability when recovering sets can intersect in a small number of coordinates. This feature allows us to increase the achievable code rate and still meet load balancing requirements. In this paper we derived an upper bound for the rate of such codes and gave explicit constructions of codes with such a property.

Acknowledgment

A. Frolov thanks A. Barg for introducing this problem to him and for numerous fruitful discussions during his stay in University of Maryland.