A Generic Transformation for Optimal Node Repair in MDS Array Codes over   $\mathbf{F}_2$

Jie Li; Xiaohu Tang; and Camilla Hollanti

arXiv:1901.06067·cs.IT·November 9, 2021

A Generic Transformation for Optimal Node Repair in MDS Array Codes over $\mathbf{F}_2$

Jie Li, Xiaohu Tang, and Camilla Hollanti

PDF

TL;DR

This paper introduces a new generic transformation for binary MDS array codes that enables optimal repair bandwidth and rebuilding access for all nodes, addressing limitations of previous transformations that did not work for binary codes.

Contribution

A novel transformation method for binary MDS array codes that ensures optimal repair properties for all nodes while preserving existing efficiencies.

Findings

01

Transformation applies to any [n,k] binary MDS array code.

02

Multiple applications yield codes with optimal rebuilding access for all nodes.

03

Existing codes with partial optimality can be extended to full optimality.

Abstract

For high-rate linear systematic maximum distance separable (MDS) codes, most early constructions could initially optimally repair all the systematic nodes but not all the parity nodes. Fortunately, this issue was first solved by Li et al. in (IEEE Trans. Inform. Theory, 64(9), 6257-6267, 2018), where a transformation that can convert any nonbinary MDS array code into another one with desired properties was proposed. However, the transformation does not work for binary MDS array codes. In this paper, we address this issue by proposing another generic transformation that can convert any $[n, k]$ binary MDS array code into a new one, which endows any $r = n - k \geq 2$ chosen nodes with optimal repair bandwidth and optimal rebuilding access properties, and at the same time, preserves the normalized repair bandwidth/rebuilding access for the remaining $k$ nodes under some conditions. As two…

Tables14

Table 1. TABLE I: A [ 9 , 6 ] 9 6 [9,6] MDS array code with optimal rebuilding access for the target nodes

RN 0 ( $𝐟_{0}$ )	$\dots$	RN $5$ ( $𝐟_{5}$ )	TN $0$ ( $𝐡_{0}$ )	TN $1$ ( $𝐡_{1}$ )	TN $2$ ( $𝐡_{2}$ )
$𝐟_{0}^{(0)}$	$\dots$	$𝐟_{5}^{(0)}$	$𝐠_{0, 0}^{(0)}$	$𝐠_{1, 0}^{(0)} + 𝐠_{0, 0}^{(1)}$	$𝐠_{2, 0}^{(0)} + 𝐠_{0, 0}^{(2)}$
$𝐟_{0}^{(0)}$	$\dots$	$𝐟_{5}^{(0)}$	$𝐠_{0, 1}^{(0)}$	$𝐠_{1, 1}^{(0)} + 𝐠_{0, 1}^{(1)}$	$𝐠_{2, 1}^{(0)} + 𝐠_{0, 1}^{(2)}$
$𝐟_{0}^{(1)}$	$\dots$	$𝐟_{5}^{(1)}$	$𝐠_{0, 0}^{(1)} + 𝐠_{1, 0}^{(0)} + 𝐠_{1, 1}^{(0)}$	$𝐠_{1, 0}^{(1)}$	$𝐠_{2, 0}^{(1)} + 𝐠_{1, 0}^{(2)}$
$𝐟_{0}^{(1)}$	$\dots$	$𝐟_{5}^{(1)}$	$𝐠_{0, 1}^{(1)} + 𝐠_{1, 0}^{(0)}$	$𝐠_{1, 1}^{(1)}$	$𝐠_{2, 1}^{(1)} + 𝐠_{1, 1}^{(2)}$
$𝐟_{0}^{(2)}$	$\dots$	$𝐟_{5}^{(2)}$	$𝐠_{0, 0}^{(2)} + 𝐠_{2, 0}^{(0)} + 𝐠_{2, 1}^{(0)}$	$𝐠_{1, 0}^{(2)} + 𝐠_{2, 0}^{(1)} + 𝐠_{2, 1}^{(1)}$	$𝐠_{2, 0}^{(2)}$
$𝐟_{0}^{(2)}$	$\dots$	$𝐟_{5}^{(2)}$	$𝐠_{0, 1}^{(2)} + 𝐠_{2, 0}^{(0)}$	$𝐠_{1, 1}^{(2)} + 𝐠_{2, 0}^{(1)}$	$𝐠_{2, 1}^{(2)}$

Table 2. TABLE II: The new code 𝒞 4 subscript 𝒞 4 \mathcal{C}_{4}

RN 0	$\dots$	RN $k - 1$	TN $0$ ( $𝐡^{'}_{0}$ )	TN $1$ ( $𝐡^{'}_{1}$ )	$\dots$	TN $r - 1$ ( $𝐡^{'}_{r - 1}$ )
$𝐟_{0}^{(0)}$	$\dots$	$𝐟_{k - 1}^{(0)}$	$𝐡_{0}^{(0)}$	$𝐡_{1}^{(0)} + 𝐡_{0}^{(1)}$	$\dots$	$𝐡_{r - 1}^{(0)} + 𝐡_{0}^{(r - 1)}$
$𝐟_{0}^{(1)}$	$\dots$	$𝐟_{k - 1}^{(1)}$	$𝐡_{0}^{(1)} ⊞_{N} 𝐡_{1}^{(0)}$	$𝐡_{1}^{(1)}$	$\dots$	$𝐡_{r - 1}^{(1)} + 𝐡_{1}^{(r - 1)}$
$⋮$	$⋱$	$⋮$	$⋮$	$⋮$	$⋱$	$⋮$
$𝐟_{0}^{(r - 1)}$	$\dots$	$𝐟_{k - 1}^{(r - 1)}$	$𝐡_{0}^{(r - 1)} ⊞_{N} 𝐡_{r - 1}^{(0)}$	$𝐡_{1}^{(r - 1)} ⊞_{N} 𝐡_{r - 1}^{(1)}$	$\dots$	$𝐡_{r - 1}^{(r - 1)}$

Table 3. TABLE III: Data from the t 𝑡 t connected target nodes

TN $j_{0}$	TN $j_{1}$	$\dots$	TN $j_{t - 1}$
$𝐡_{j_{0}}^{(j_{0})}$	$𝐡_{j_{1}}^{(j_{0})} + 𝐡_{j_{0}}^{(j_{1})}$	$\dots$	$𝐡_{j_{t - 1}}^{(j_{0})} + 𝐡_{j_{0}}^{(j_{t - 1})}$
$𝐡_{j_{0}}^{(j_{1})} ⊞_{N} 𝐡_{j_{1}}^{(j_{0})}$	$𝐡_{j_{1}}^{(j_{1})}$	$\dots$	$𝐡_{j_{t - 1}}^{(j_{1})} + 𝐡_{j_{1}}^{(j_{t - 1})}$
$⋮$	$⋮$	$⋱$	$⋮$
$𝐡_{j_{0}}^{(j_{t - 1})} ⊞_{N} 𝐡_{j_{t - 1}}^{(j_{0})}$	$𝐡_{j_{1}}^{(j_{t - 1})} ⊞_{N} 𝐡_{j_{t - 1}}^{(j_{1})}$	$\dots$	$𝐡_{j_{t - 1}}^{(j_{t - 1})}$

Table 4. TABLE IV: A part of data at the target nodes that are connected, where 𝐡 j a ( j b ) ⋄ 𝐡 j a ( j b ) ⋄ superscript subscript 𝐡 subscript 𝑗 𝑎 subscript 𝑗 𝑏 superscript subscript 𝐡 subscript 𝑗 𝑎 subscript 𝑗 𝑏 \mathbf{h}_{j_{a}}^{(j_{b})}~{}\diamond~{}{\mathbf{h}_{j_{a}}^{(j_{b})}} denotes 𝐡 j a ( j b ) ⊞ N 𝐡 j b ( j a ) subscript ⊞ 𝑁 superscript subscript 𝐡 subscript 𝑗 𝑎 subscript 𝑗 𝑏 superscript subscript 𝐡 subscript 𝑗 𝑏 subscript 𝑗 𝑎 \mathbf{h}_{j_{a}}^{(j_{b})}\boxplus_{N}{\mathbf{h}_{j_{b}}^{(j_{a})}} if j a < j b subscript 𝑗 𝑎 subscript 𝑗 𝑏 j_{a}<j_{b} and 𝐡 j a ( j b ) + 𝐡 j b ( j a ) superscript subscript 𝐡 subscript 𝑗 𝑎 subscript 𝑗 𝑏 superscript subscript 𝐡 subscript 𝑗 𝑏 subscript 𝑗 𝑎 \mathbf{h}_{j_{a}}^{(j_{b})}+{\mathbf{h}_{j_{b}}^{(j_{a})}} if j a > j b subscript 𝑗 𝑎 subscript 𝑗 𝑏 j_{a}>j_{b}

TN $j_{0}$	TN $j_{1}$	$\dots$	TN $j_{t - 1}$
$𝐡_{j_{0}}^{(j_{t})} ⋄ 𝐡_{j_{t}}^{(j_{0})}$	$𝐡_{j_{1}}^{(j_{t})} ⋄ 𝐡_{j_{t}}^{(j_{1})}$	$\dots$	$𝐡_{j_{t - 1}}^{(j_{t})} ⋄ 𝐡_{j_{t}}^{(j_{t - 1})}$
$⋮$	$⋮$	$⋱$	$⋮$
$𝐡_{j_{0}}^{(j_{r - 1})} ⋄ 𝐡_{j_{r - 1}}^{(j_{0})}$	$𝐡_{j_{1}}^{(j_{r - 1})} ⋄ 𝐡_{j_{r - 1}}^{(j_{1})}$	$\dots$	$𝐡_{j_{t - 1}}^{(j_{r - 1})} ⋄ 𝐡_{j_{r - 1}}^{(j_{t - 1})}$

Table 5. TABLE V: Structure of the new storage code 𝒞 4 ′ superscript subscript 𝒞 4 ′ \mathcal{C}_{4}^{\prime}

RN 0	$\dots$	RN $r - 1$	RN $r$	$\dots$	RN $k - 1$	TN $0$	$\dots$	TN $r - 1$
$𝐟^{'}_{0}^{(0)}$	$\dots$	$𝐟^{'}_{r - 1}^{(0)}$	$𝐟_{r}^{(0)}$	$\dots$	$𝐟_{k - 1}^{(0)}$	$𝐡_{0}^{(0)}$	$\dots$	$𝐡_{r - 1}^{(0)}$
$𝐟^{'}_{0}^{(1)}$	$\dots$	$𝐟^{'}_{r - 1}^{(1)}$	$𝐟_{r}^{(1)}$	$\dots$	$𝐟_{k - 1}^{(1)}$	$𝐡_{0}^{(1)}$	$\dots$	$𝐡_{r - 1}^{(1)}$
$⋮$	$⋱$	$⋮$	$⋮$	$⋱$	$⋮$	$⋮$	$⋱$	$⋮$
$𝐟^{'}_{0}^{(r - 1)}$	$\dots$	$𝐟^{'}_{r - 1}^{(r - 1)}$	$𝐟_{r}^{(r - 1)}$	$\dots$	$𝐟_{k - 1}^{(r - 1)}$	$𝐡_{0}^{(r - 1)}$	$\dots$	$𝐡_{r - 1}^{(r - 1)}$

Table 6. TABLE VI: The storage code 𝒞 3 ′ superscript subscript 𝒞 3 ′ \mathcal{C}_{3}^{\prime}

RN 0	$\dots$	RN $r - 1$	RN $r$	$\dots$	RN $k - 1$	TN $0$	$\dots$	TN $r - 1$
$𝐟^{'}_{0}^{(0)}$	$\dots$	$𝐟^{'}_{r - 1}^{(0)}$	$𝐟_{r}^{(0)}$	$\dots$	$𝐟_{k - 1}^{(0)}$	$𝐯_{0}^{(0)}$	$\dots$	$𝐯_{r - 1}^{(0)}$
$𝐟^{'}_{0}^{(1)}$	$\dots$	$𝐟^{'}_{r - 1}^{(1)}$	$𝐟_{r}^{(1)}$	$\dots$	$𝐟_{k - 1}^{(1)}$	$𝐯_{0}^{(1)}$	$\dots$	$𝐯_{r - 1}^{(1)}$
$⋮$	$⋱$	$⋮$	$⋮$	$⋱$	$⋮$	$⋮$	$⋱$	$⋮$
$𝐟^{'}_{0}^{(r - 1)}$	$\dots$	$𝐟^{'}_{r - 1}^{(r - 1)}$	$𝐟_{r}^{(r - 1)}$	$\dots$	$𝐟_{k - 1}^{(r - 1)}$	$𝐯_{0}^{(r - 1)}$	$\dots$	$𝐯_{r - 1}^{(r - 1)}$

Table 7. TABLE VII : A [ 5 , 3 ] 5 3 [5,3] EVENODD code 𝒬 0 subscript 𝒬 0 \mathcal{Q}_{0} , where SN and PN denote systematic node and parity node, respectively

SN $0$	SN $1$	SN $2$	PN $0$	PN $1$
$a_{0}$	$b_{0}$	$c_{0}$	$a_{0} + b_{0} + c_{0}$	$a_{0} + b_{1} + c_{0} + c_{1}$
$a_{1}$	$b_{1}$	$c_{1}$	$a_{1} + b_{1} + c_{1}$	$a_{1} + b_{0} + b_{1} + c_{0}$

Table 8. TABLE VIII : A [ 5 , 3 ] 5 3 [5,3] binary MDS array code 𝒬 1 subscript 𝒬 1 \mathcal{Q}_{1} , where systematic nodes 0 and 1 are chosen as the target nodes

SN $0$	SN $1$	SN $2$	PN $0$	PN $1$
$a_{0}$	$b_{0}$	$c_{0}$	$a_{0} + a_{2} + a_{3} + b_{0} + b_{1} + c_{0}$	$a_{0} + a_{2} + b_{0} + c_{0} + c_{1}$
$a_{1}$	$b_{1}$	$c_{1}$	$a_{1} + a_{2} + b_{0} + c_{1}$	$a_{1} + a_{3} + b_{1} + c_{0}$
$a_{2}$	$b_{2}$	$c_{2}$	$a_{2} + a_{3} + b_{1} + b_{2} + c_{2}$	$a_{2} + a_{3} + b_{1} + b_{3} + c_{2} + c_{3}$
$a_{3}$	$b_{3}$	$c_{3}$	$a_{2} + b_{0} + b_{1} + b_{3} + c_{3}$	$a_{2} + b_{0} + b_{1} + b_{2} + b_{3} + c_{2}$

Table 9. TABLE IX : The [ 5 , 3 ] 5 3 [5,3] binary MDS array code 𝒬 2 subscript 𝒬 2 \mathcal{Q}_{2} in systematic form, where systematic nodes 1 and 2 are chosen as the target nodes

SN $0$	SN $1$	SN $2$	PN $0$	PN $1$
$a_{0}$	$b_{0}$	$c_{0}$	$a_{0} + a_{2} + a_{3} + b_{0} + b_{1} + b_{4} + b_{5} + c_{0} + c_{1}$	$a_{0} + a_{2} + b_{0} + b_{5} + c_{1}$
$a_{1}$	$b_{1}$	$c_{1}$	$a_{1} + a_{2} + b_{0} + b_{4} + c_{0}$	$a_{1} + a_{3} + b_{1} + b_{4} + b_{5} + c_{0} + c_{1}$
$a_{2}$	$b_{2}$	$c_{2}$	$a_{2} + a_{3} + b_{1} + b_{2} + b_{6} + b_{7} + c_{2} + c_{3}$	$a_{2} + a_{3} + b_{1} + b_{3} + b_{7} + c_{3}$
$a_{3}$	$b_{3}$	$c_{3}$	$a_{2} + b_{0} + b_{1} + b_{3} + b_{6} + c_{2}$	$a_{2} + b_{0} + b_{1} + b_{2} + b_{3} + b_{6} + b_{7} + c_{2} + c_{3}$
$a_{4}$	$b_{4}$	$c_{4}$	$a_{4} + a_{6} + a_{7} + b_{5} + c_{0} + c_{4}$	$a_{4} + a_{6} + b_{4} + b_{5} + c_{1} + c_{4} + c_{5}$
$a_{5}$	$b_{5}$	$c_{5}$	$a_{5} + a_{6} + b_{4} + b_{5} + c_{1} + c_{5}$	$a_{5} + a_{7} + b_{4} + c_{0} + c_{1} + c_{4}$
$a_{6}$	$b_{6}$	$c_{6}$	$a_{6} + a_{7} + b_{4} + b_{6} + b_{7} + c_{0} + c_{1} + c_{3} + c_{6}$	$a_{6} + a_{7} + b_{4} + b_{6} + c_{0} + c_{1} + c_{2} + c_{3} + c_{6} + c_{7}$
$a_{7}$	$b_{7}$	$c_{7}$	$a_{6} + b_{5} + b_{6} + c_{0} + c_{2} + c_{3} + c_{7}$	$a_{6} + b_{5} + b_{7} + c_{0} + c_{2} + c_{6}$

Table 10. TABLE X : An intermediate code 𝒬 0 ′ subscript superscript 𝒬 ′ 0 \mathcal{Q}^{\prime}_{0} , where SN and PN respectively denote systematic node and parity node

SN $0$	SN $1$	SN $2$	PN $0$	PN $1$
$a_{0}$	$b_{0}$	$c_{0}$	$a_{0} + b_{0} + c_{0}$	$a_{0} + b_{1} + c_{0} + c_{1}$
$a_{1}$	$b_{1}$	$c_{1}$	$a_{1} + b_{1} + c_{1}$	$a_{1} + b_{0} + b_{1} + c_{0}$
$a_{2}$	$b_{2}$	$c_{2}$	$a_{2} + b_{2} + c_{2}$	$a_{2} + b_{3} + c_{2} + c_{3}$
$a_{3}$	$b_{3}$	$c_{3}$	$a_{3} + b_{3} + c_{3}$	$a_{3} + b_{2} + b_{3} + c_{2}$

Table 11. TABLE XI: The [ 6 , 4 ] 6 4 [6,4] MDR-1 code 𝒞 1 subscript 𝒞 1 \mathcal{C}_{1} over 𝐅 2 subscript 𝐅 2 \mathbf{F}_{2}

SN $0$	SN $1$	SN $2$	SN $3$	PN $0$	PN $1$
$a_{0}$	$b_{0}$	$c_{0}$	$d_{0}$	$a_{0} + b_{0} + c_{0} + d_{0}$	$a_{0} + a_{3} + b_{0} + c_{1} + c_{4} + d_{4}$
$a_{1}$	$b_{1}$	$c_{1}$	$d_{1}$	$a_{1} + b_{1} + c_{1} + d_{1}$	$a_{1} + a_{2} + b_{1} + c_{5} + d_{0} + d_{5}$
$a_{2}$	$b_{2}$	$c_{2}$	$d_{2}$	$a_{2} + b_{2} + c_{2} + d_{2}$	$a_{2} + b_{1} + b_{2} + c_{6} + d_{3} + d_{6}$
$a_{3}$	$b_{3}$	$c_{3}$	$d_{3}$	$a_{3} + b_{3} + c_{3} + d_{3}$	$a_{3} + b_{0} + b_{3} + c_{2} + c_{7} + d_{7}$
$a_{4}$	$b_{4}$	$c_{4}$	$d_{4}$	$a_{4} + b_{4} + c_{4} + d_{4}$	$a_{0} + a_{4} + a_{7} + b_{0} + b_{4} + c_{0} + c_{5} + d_{0}$
$a_{5}$	$b_{5}$	$c_{5}$	$d_{5}$	$a_{5} + b_{5} + c_{5} + d_{5}$	$a_{1} + a_{5} + a_{6} + b_{1} + b_{5} + c_{1} + d_{1} + d_{4}$
$a_{6}$	$b_{6}$	$c_{6}$	$d_{6}$	$a_{6} + b_{6} + c_{6} + d_{6}$	$a_{2} + a_{6} + b_{2} + b_{5} + b_{6} + c_{2} + d_{2} + d_{7}$
$a_{7}$	$b_{7}$	$c_{7}$	$d_{7}$	$a_{7} + b_{7} + c_{7} + d_{7}$	$a_{3} + a_{7} + b_{3} + b_{4} + b_{7} + c_{3} + c_{6} + d_{3}$

Table 12. TABLE XII : A [ 6 , 4 ] 6 4 [6,4] binary MDS array code 𝒞 3 subscript 𝒞 3 \mathcal{C}_{3}

SN $0$	SN $1$	SN $2$	SN $3$	PN $0$	PN $1$
$f_{0}^{(0)}$	$f_{1}^{(0)}$	$f_{2}^{(0)}$	$f_{3}^{(0)}$	$g_{0}^{(0)}$	$g_{1}^{(0)} + g_{0}^{(2)}$
$f_{0}^{(1)}$	$f_{1}^{(1)}$	$f_{2}^{(1)}$	$f_{3}^{(1)}$	$g_{0}^{(1)}$	$g_{1}^{(1)} + g_{0}^{(3)}$
$f_{0}^{(2)}$	$f_{1}^{(2)}$	$f_{2}^{(2)}$	$f_{3}^{(2)}$	$g_{0}^{(2)} + g_{1}^{(0)} + g_{1}^{(1)}$	$g_{1}^{(2)}$
$f_{0}^{(3)}$	$f_{1}^{(3)}$	$f_{2}^{(3)}$	$f_{3}^{(3)}$	$g_{0}^{(3)} + g_{1}^{(0)}$	$g_{1}^{(3)}$

Table 13. TABLE XIII: Comparison of the (extra) complexity between the transformations obtained in this paper and the one in [ 35 ] (based on the 1st transformation in Section II), where t 𝑡 t denotes the number of target nodes that are connected during the reconstruction

	Complexity for Pairing	Extra complexity for reconstruction	Extra complexity for repairing TN
The transformation in [35]	$2 r (r - 1) α^{'}$ XORs	$\frac{1}{2} (6 r t - 3 t^{2} - 3 t) α^{'}$ XORs	$4 (r - 1) α^{'}$ XORs
The proposed transformation	$\frac{5 r (r - 1)}{4} α^{'}$ XORs	$\frac{1}{2} (3 r t - t^{2} - 2 t) α^{'}$ XORs	$\frac{5}{2} (r - 1) α^{'}$ XORs

Table 14. TABLE XIV: Comparison between the transformations obtained in this paper and the one in [ 35 ]

	Pairing technique involved	Computation complexity	Codes can be applied
The transformation in [35]	Case-by-case	High	Some specific codes
The proposed transformation	Uniform	Low	Any binary MDS array codes

Equations94

γ^{*} (n - 1) = \frac{n - 1}{n - k} α .

γ^{*} (n - 1) = \frac{n - 1}{n - k} α .

\displaystyle\mathbf{f}_{i}^{(l)}=\left(\begin{array}[]{c}\mathbf{f}_{i,0}^{(l)}\\ \mathbf{f}_{i,1}^{(l)}\\ \end{array}\right),~{}\mathbf{g}_{j}^{(l)}=\left(\begin{array}[]{c}\mathbf{g}_{j,0}^{(l)}\\ \mathbf{g}_{j,1}^{(l)}\\ \end{array}\right).

\displaystyle\mathbf{f}_{i}^{(l)}=\left(\begin{array}[]{c}\mathbf{f}_{i,0}^{(l)}\\ \mathbf{f}_{i,1}^{(l)}\\ \end{array}\right),~{}\mathbf{g}_{j}^{(l)}=\left(\begin{array}[]{c}\mathbf{g}_{j,0}^{(l)}\\ \mathbf{g}_{j,1}^{(l)}\\ \end{array}\right).

\mathbf{h}_{i}=\left(\begin{array}[]{c}\mathbf{h}_{i}^{(0)}\\ \mathbf{h}_{i}^{(1)}\\ \mathbf{h}_{i}^{(2)}\\ \end{array}\right)~{}~{}\mbox{and}~{}~{}\mathbf{h}_{i}^{(l)}=\left(\begin{array}[]{c}\mathbf{h}_{i,0}^{(l)}\\ \mathbf{h}_{i,1}^{(l)}\\ \end{array}\right),

\mathbf{h}_{i}=\left(\begin{array}[]{c}\mathbf{h}_{i}^{(0)}\\ \mathbf{h}_{i}^{(1)}\\ \mathbf{h}_{i}^{(2)}\\ \end{array}\right)~{}~{}\mbox{and}~{}~{}\mathbf{h}_{i}^{(l)}=\left(\begin{array}[]{c}\mathbf{h}_{i,0}^{(l)}\\ \mathbf{h}_{i,1}^{(l)}\\ \end{array}\right),

\mathbf{h}_{i,j}^{(l)}=\left\{\begin{array}[]{ll}\mathbf{g}_{i,j}^{(i)},&\textrm{if}~{}i=l,\\ \mathbf{g}_{i,0}^{(l)}+\mathbf{g}_{l,0}^{(i)}+\mathbf{g}_{l,1}^{(i)},&\textrm{if}~{}i<l~{}\textrm{and}~{}j=0,\\ \mathbf{g}_{i,1}^{(l)}+\mathbf{g}_{l,0}^{(i)},&\textrm{if}~{}i<l~{}\textrm{and}~{}j=1,\\ \mathbf{g}_{i,j}^{(l)}+\mathbf{g}_{l,j}^{(i)},&\textrm{if}~{}i>l,\end{array}\right.

\mathbf{h}_{i,j}^{(l)}=\left\{\begin{array}[]{ll}\mathbf{g}_{i,j}^{(i)},&\textrm{if}~{}i=l,\\ \mathbf{g}_{i,0}^{(l)}+\mathbf{g}_{l,0}^{(i)}+\mathbf{g}_{l,1}^{(i)},&\textrm{if}~{}i<l~{}\textrm{and}~{}j=0,\\ \mathbf{g}_{i,1}^{(l)}+\mathbf{g}_{l,0}^{(i)},&\textrm{if}~{}i<l~{}\textrm{and}~{}j=1,\\ \mathbf{g}_{i,j}^{(l)}+\mathbf{g}_{l,j}^{(i)},&\textrm{if}~{}i>l,\end{array}\right.

g_{1, 1}^{(0)} = h_{0, 0}^{(1)} + h_{1, 0}^{(0)}, g_{0, 1}^{(1)} = g_{1, 1}^{(0)} + h_{1, 1}^{(0)},

g_{1, 1}^{(0)} = h_{0, 0}^{(1)} + h_{1, 0}^{(0)}, g_{0, 1}^{(1)} = g_{1, 1}^{(0)} + h_{1, 1}^{(0)},

g_{1, 0}^{(0)} = g_{0, 1}^{(1)} + h_{0, 1}^{(1)}, g_{0, 0}^{(1)} = g_{1, 0}^{(0)} + h_{1, 0}^{(0)} .

(f_{2}^{(0)}, \dots, f_{5}^{(0)}, g_{0}^{(0)}, g_{1}^{(0)}), (f_{2}^{(1)}, \dots, f_{5}^{(1)}, g_{0}^{(1)}, g_{1}^{(1)}),

(f_{2}^{(0)}, \dots, f_{5}^{(0)}, g_{0}^{(0)}, g_{1}^{(0)}), (f_{2}^{(1)}, \dots, f_{5}^{(1)}, g_{0}^{(1)}, g_{1}^{(1)}),

\mathbf{g}_{0}^{(2)}=\left(\begin{array}[]{c}\mathbf{g}_{0,0}^{(2)}\\ \mathbf{g}_{0,1}^{(2)}\\ \end{array}\right)=\left(\begin{array}[]{l}\mathbf{h}_{0,0}^{(2)}+\mathbf{g}_{2,0}^{(0)}+\mathbf{g}_{2,1}^{(0)}\\ \mathbf{h}_{0,1}^{(2)}+\mathbf{g}_{2,0}^{(0)}\\ \end{array}\right),

\mathbf{g}_{0}^{(2)}=\left(\begin{array}[]{c}\mathbf{g}_{0,0}^{(2)}\\ \mathbf{g}_{0,1}^{(2)}\\ \end{array}\right)=\left(\begin{array}[]{l}\mathbf{h}_{0,0}^{(2)}+\mathbf{g}_{2,0}^{(0)}+\mathbf{g}_{2,1}^{(0)}\\ \mathbf{h}_{0,1}^{(2)}+\mathbf{g}_{2,0}^{(0)}\\ \end{array}\right),

\mathbf{g}_{1}^{(2)}=\left(\begin{array}[]{c}\mathbf{g}_{1,0}^{(2)}\\ \mathbf{g}_{1,1}^{(2)}\\ \end{array}\right)=\left(\begin{array}[]{l}\mathbf{h}_{1,0}^{(2)}+\mathbf{g}_{2,0}^{(1)}+\mathbf{g}_{2,1}^{(1)}\\ \mathbf{h}_{1,1}^{(2)}+\mathbf{g}_{2,0}^{(1)}\\ \end{array}\right).

\mathbf{g}_{1}^{(2)}=\left(\begin{array}[]{c}\mathbf{g}_{1,0}^{(2)}\\ \mathbf{g}_{1,1}^{(2)}\\ \end{array}\right)=\left(\begin{array}[]{l}\mathbf{h}_{1,0}^{(2)}+\mathbf{g}_{2,0}^{(1)}+\mathbf{g}_{2,1}^{(1)}\\ \mathbf{h}_{1,1}^{(2)}+\mathbf{g}_{2,0}^{(1)}\\ \end{array}\right).

(f_{2}^{(2)}, \dots, f_{5}^{(2)}, g_{0}^{(2)}, g_{2}^{(2)}),

(f_{2}^{(2)}, \dots, f_{5}^{(2)}, g_{0}^{(2)}, g_{2}^{(2)}),

f_{0}^{(0)}, \dots, f_{5}^{(0)}, h_{1}^{(0)}, h_{2}^{(0)},

f_{0}^{(0)}, \dots, f_{5}^{(0)}, h_{1}^{(0)}, h_{2}^{(0)},

\mathbf{h}_{0}^{(1)}=\left(\begin{array}[]{l}\mathbf{g}_{0,0}^{(1)}+\mathbf{g}_{1,0}^{(0)}+\mathbf{g}_{1,1}^{(0)}\\ \mathbf{g}_{0,1}^{(1)}+\mathbf{g}_{1,0}^{(0)}\end{array}\right)

\mathbf{h}_{0}^{(1)}=\left(\begin{array}[]{l}\mathbf{g}_{0,0}^{(1)}+\mathbf{g}_{1,0}^{(0)}+\mathbf{g}_{1,1}^{(0)}\\ \mathbf{g}_{0,1}^{(1)}+\mathbf{g}_{1,0}^{(0)}\end{array}\right)

\displaystyle\mathbf{a}[i]=\left(\begin{array}[]{c}\mathbf{a}[i,0]\\ \mathbf{a}[i,1]\\ \end{array}\right),~{}~{}\mathbf{b}[i]=\left(\begin{array}[]{c}\mathbf{b}[i,0]\\ \mathbf{b}[i,1]\\ \end{array}\right).

\displaystyle\mathbf{a}[i]=\left(\begin{array}[]{c}\mathbf{a}[i,0]\\ \mathbf{a}[i,1]\\ \end{array}\right),~{}~{}\mathbf{b}[i]=\left(\begin{array}[]{c}\mathbf{b}[i,0]\\ \mathbf{b}[i,1]\\ \end{array}\right).

a [i] ⊞ b [i]

a [i] ⊞ b [i]

=

\displaystyle\mathbf{a}=\left(\begin{array}[]{c}\mathbf{a}{[0]}\\ \mathbf{a}{[1]}\\ \vdots\\ \mathbf{a}{[\delta-1]}\\ \end{array}\right),~{}~{}\mathbf{b}=\left(\begin{array}[]{c}\mathbf{b}{[0]}\\ \mathbf{b}{[1]}\\ \vdots\\ \mathbf{b}{[\delta-1]}\\ \end{array}\right),

\displaystyle\mathbf{a}=\left(\begin{array}[]{c}\mathbf{a}{[0]}\\ \mathbf{a}{[1]}\\ \vdots\\ \mathbf{a}{[\delta-1]}\\ \end{array}\right),~{}~{}\mathbf{b}=\left(\begin{array}[]{c}\mathbf{b}{[0]}\\ \mathbf{b}{[1]}\\ \vdots\\ \mathbf{b}{[\delta-1]}\\ \end{array}\right),

\displaystyle\mathbf{a}\boxplus_{N}\mathbf{b}=\left(\begin{array}[]{c}\mathbf{a}{[0]}\boxplus\mathbf{b}{[0]}\\ \mathbf{a}{[1]}\boxplus\mathbf{b}{[1]}\\ \vdots\\ \mathbf{a}{[\delta-1]}\boxplus\mathbf{b}{[\delta-1]}\\ \end{array}\right),

\displaystyle\mathbf{a}\boxplus_{N}\mathbf{b}=\left(\begin{array}[]{c}\mathbf{a}{[0]}\boxplus\mathbf{b}{[0]}\\ \mathbf{a}{[1]}\boxplus\mathbf{b}{[1]}\\ \vdots\\ \mathbf{a}{[\delta-1]}\boxplus\mathbf{b}{[\delta-1]}\\ \end{array}\right),

\mathbf{h}_{j}=\left(\begin{array}[]{c}\mathbf{h}_{j}^{(0)}\\ \vdots\\ \mathbf{h}_{j}^{(r-1)}\\ \end{array}\right),j\in[0,r)

\mathbf{h}_{j}=\left(\begin{array}[]{c}\mathbf{h}_{j}^{(0)}\\ \vdots\\ \mathbf{h}_{j}^{(r-1)}\\ \end{array}\right),j\in[0,r)

h_{j}^{(l)} = g_{π_{l} (j)}^{(l)}, j, l \in [0, r),

h_{j}^{(l)} = g_{π_{l} (j)}^{(l)}, j, l \in [0, r),

\mathbf{h^{\prime}}_{j}=\left(\begin{array}[]{c}\mathbf{h^{\prime}}_{j}^{(0)}\\ \vdots\\ \mathbf{h^{\prime}}_{j}^{(r-1)}\\ \end{array}\right)

\mathbf{h^{\prime}}_{j}=\left(\begin{array}[]{c}\mathbf{h^{\prime}}_{j}^{(0)}\\ \vdots\\ \mathbf{h^{\prime}}_{j}^{(r-1)}\\ \end{array}\right)

\displaystyle\mathbf{h^{\prime}}_{j}^{(l)}=\left\{\begin{array}[]{ll}\mathbf{h}_{j}^{(j)},&\textrm{if}~{}j=l,\\ \mathbf{h}_{j}^{(l)}+\mathbf{h}_{l}^{(j)},&\textrm{if}~{}j>l,\\ \mathbf{h}_{j}^{(l)}\boxplus_{N}\mathbf{h}_{l}^{(j)},&\textrm{if}~{}j<l,\end{array}\right.

\displaystyle\mathbf{h^{\prime}}_{j}^{(l)}=\left\{\begin{array}[]{ll}\mathbf{h}_{j}^{(j)},&\textrm{if}~{}j=l,\\ \mathbf{h}_{j}^{(l)}+\mathbf{h}_{l}^{(j)},&\textrm{if}~{}j>l,\\ \mathbf{h}_{j}^{(l)}\boxplus_{N}\mathbf{h}_{l}^{(j)},&\textrm{if}~{}j<l,\end{array}\right.

b l k d ia g (S_{i, k + j, 0} S_{i, k + j, 0} \dots S_{i, k + j, δ - 1} S_{i, k + j, δ - 1})

b l k d ia g (S_{i, k + j, 0} S_{i, k + j, 0} \dots S_{i, k + j, δ - 1} S_{i, k + j, δ - 1})

\left(\begin{array}[]{cc}S_{i,k+j,m}&\\ &S_{i,k+j,m}\\ \end{array}\right)\mathbf{g}_{j}^{(l)}[m],j\in[0,r)

\left(\begin{array}[]{cc}S_{i,k+j,m}&\\ &S_{i,k+j,m}\\ \end{array}\right)\mathbf{g}_{j}^{(l)}[m],j\in[0,r)

S_{i, k + π_{l} (j), m} = S_{i, k + π_{j} (l), m}

S_{i, k + π_{l} (j), m} = S_{i, k + π_{j} (l), m}

\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(j),m}&\\ &S_{i,k+\pi_{l}(j),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{j}^{(l)}[m]

\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(j),m}&\\ &S_{i,k+\pi_{l}(j),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{j}^{(l)}[m]

\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(j),m}&\\ &S_{i,k+\pi_{l}(j),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{j}^{(l)}[m]

\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(j),m}&\\ &S_{i,k+\pi_{l}(j),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{j}^{(l)}[m]

\left(\begin{array}[]{cc}S_{i,k+\pi_{j}(l),m}&\\ &S_{i,k+\pi_{j}(l),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{l}^{(j)}[m],

\left(\begin{array}[]{cc}S_{i,k+\pi_{j}(l),m}&\\ &S_{i,k+\pi_{j}(l),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{l}^{(j)}[m],

\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(j),m}&\\ &S_{i,k+\pi_{l}(j),m}\\ \end{array}\right)\mathbf{h}_{j}^{(l)}[m]

\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(j),m}&\\ &S_{i,k+\pi_{l}(j),m}\\ \end{array}\right)\mathbf{h}_{j}^{(l)}[m]

\left(\begin{array}[]{cc}S_{i,k+\pi_{j}(l),m}&\\ &S_{i,k+\pi_{j}(l),m}\\ \end{array}\right)\mathbf{h}_{l}^{(j)}[m]

\left(\begin{array}[]{cc}S_{i,k+\pi_{j}(l),m}&\\ &S_{i,k+\pi_{j}(l),m}\\ \end{array}\right)\mathbf{h}_{l}^{(j)}[m]

\displaystyle\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(l),m}&\\ &S_{i,k+\pi_{l}(l),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{l}^{(l)}[m]

\displaystyle\left(\begin{array}[]{cc}S_{i,k+\pi_{l}(l),m}&\\ &S_{i,k+\pi_{l}(l),m}\\ \end{array}\right)\mathbf{h^{\prime}}_{l}^{(l)}[m]

=

\left(\begin{array}[]{cc}S_{i,k+j,m}&\\ &S_{i,k+j,m}\\ \end{array}\right)\mathbf{g}_{j}^{(l)}[m],m\in[0,\delta),

Peer Reviews

No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.

Videos

No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.

Full text

A Generic Transformation for Optimal Node Repair in MDS Array Codes over $\mathbf{F}_{2}$

Jie Li, Xiaohu Tang, and Camilla Hollanti, The work of J. Li was supported in part by the National Science Foundation of China under Grant No. 61801176. The work of Xiaohu Tang was supported in part by the National Natural Science Foundation of China under Grant No. 61941106 and 61871331. The work of C. Hollanti was supported by the Academy of Finland, under Grants No. 318937 and 336005.J. Li was with the Hubei Key Laboratory of Applied Mathematics, Faculty of Mathematics and Statistics, Hubei University, Wuhan 430062, China, and also with the Department of Mathematics and Systems Analysis, P.O. Box 11100, Aalto University, FI-00076 Aalto (Espoo), Finland (e-mail: [email protected]).X. Tang is with the Information Security and National Computing Grid Laboratory, Southwest Jiaotong University, Chengdu 610031, China (e-mail: [email protected]).C. Hollanti is with the Department of Mathematics and Systems Analysis, P.O. Box 11100, Aalto University, FI-00076 Aalto (Espoo), Finland (e-mail: [email protected]).

Abstract

For high-rate linear systematic maximum distance separable (MDS) codes, most early constructions could initially optimally repair all the systematic nodes but not all the parity nodes. Fortunately, this issue was first solved by Li et al. in (IEEE Trans. Inform. Theory, 64(9), 6257-6267, 2018), where a transformation that can convert any nonbinary MDS array code into another one with desired properties was proposed. However, the transformation does not work for binary MDS array codes. In this paper, we address this issue by proposing another generic transformation that can convert any $[n,k]$ binary MDS array code into a new one, which endows any $r=n-k\geq 2$ chosen nodes with optimal repair bandwidth and optimal rebuilding access properties, and at the same time, preserves the normalized repair bandwidth/rebuilding access for the remaining $k$ nodes under some conditions. As two immediate applications, we show that

by applying the transformation multiple times, any binary MDS array code can be converted into one with optimal rebuilding access for all nodes, 2) any binary MDS array code with optimal repair bandwidth or optimal rebuilding access for the systematic nodes can be converted into one with the corresponding optimality property for all nodes.

Index Terms:

Binary MDS array codes, distributed storage, high-rate, optimal rebuilding access, optimal repair bandwidth.

I Introduction

In distributed storage systems, one of the major concerns is reliability. A common way to fulfill reliability is by introducing redundancy. Normally, MDS codes such as Reed-Solomon (RS) codes [1], can offer maximum reliability for a given storage overhead, thus they have been used extensively as the basis for RAID systems and distributed storage systems [1, 2, 3, 4, 5].

Upon failure of a single storage node, a self-sustaining distributed storage system must possess the ability to repair the failed node. For example, to accomplish this task, the repair process of the classical MDS codes is to first reconstruct the original file by accessing and downloading an amount of data equal to the size of the original file, and then repair the failed node, which is called naive repair. However, such repair is overly excessive, and poses the question whether we can minimize the rebuilding access and the repair bandwidth, i.e., the amount of data that needs to be accessed (or read) and downloaded to repair a failed node, respectively.

The seminal work in [6] gave a positive answer to the above question. It was shown in [6] that the minimum repair bandwidth of an $[n,k]$ MDS code $\mathcal{C}$ defined over $\mathbf{F}_{q}^{\alpha}$ (which is referred to as an array code [7] if $\alpha>1$ ) is $\gamma^{*}(d)\triangleq\frac{d}{d-k+1}\alpha$ , where $d\in[k,n)$ stands for the number of helper nodes contacted during the repair procedure. MDS codes attaining the minimum repair bandwidth are said to have optimal repair bandwidth and are referred to as minimum storage regenerating (MSR) codes. Subsequently, the optimal rebuilding access was also established in [8, 9], where an MDS code is said to possess optimal rebuilding access if the amount of data accessed meets the minimum repair bandwidth during the repair process of a failed node. Since 2010, various MDS array codes with optimal repair bandwidth have been proposed in the literature [24, 22, 19, 14, 13, 26, 21, 15, 20, 16, 17, 18, 10, 11, 12, 8, 25, 27, 23], where most works [19, 14, 13, 21, 15, 20, 16, 17, 18, 10, 11, 12] consider the case $d=n-1$ to maximally reduce the repair bandwidth, as $\gamma^{*}(d)$ is a decreasing function of $d$ ; the setting $d=n-1$ is also the focus of this work, then the optimal repair can be achieved by downloading $\frac{\alpha}{n-k}$ elements from each surviving nodes as

[TABLE]

It is worth noting that a symbol in $\mathbf{F}_{q}^{b}$ can be regarded as a symbol in $\mathbf{F}_{q^{b}}$ , and vice versa. Thus clearly, every linear code over $\mathbf{F}_{q^{b}}$ can be regarded as an $\mathbf{F}_{q}$ -linear code over $\mathbf{F}_{q}^{b}$ , but the converse is not true [28]. RS codes over $\mathbf{F}_{q^{b}}$ are examples of $\mathbf{F}_{q}$ -linear MDS array codes over $\mathbf{F}_{q}^{b}$ . Repairing RS codes has also been attracting a lot of attention recently, however, in general, they either cannot attain optimal repair bandwidth or should be constructed over a large finite field to achieve optimal repair bandwidth. Interested readers could refer to [29, 30] and the references therein for more details.

For high-rate (i.e., $k/n>1/2$ ) MDS array codes, most early systematic code constructions can only optimally repair all the systematic nodes but not all the parity nodes [14, 13, 15, 16, 17, 18, 10, 11, 12]. Fortunately, this issue was first solved by Li et al. [31, 32, 33], where a transformation that can enable optimal node repair for any $n-k$ chosen nodes of a nonbinary $[n,k]$ MDS code was proposed, and at the same time, preserves the repair efficiency for the remaining $k$ nodes. It is worth noting that independent and parallel to the works in [31, 32, 33], Ye et al. [25] and Sasidharan et al. [26] respectively proposed explicit constructions of high-rate MDS array codes, which are equivalent in essence, and have the same performance as the ones obtained from the second application in [31, 32, 33] (see [9] for the discussion on these discoveries).

The generic transformation proposed in [31, 32, 33] has wide potential applications, however, it does not work for binary MDS array codes. In this paper, we aim to address the unsolved problem in [31] by presenting a new generic transformation that can convert any binary MDS array code into another one, which endows any $r=n-k\geq 2$ chosen nodes with optimal rebuilding access, and at the same time, preserves the repair efficiency for the remaining $k$ nodes under some conditions. As two immediate applications of this transformation, we show that

any binary MDS array code can be converted into one with optimal rebuilding access for all nodes, 2) any binary MDS array code with optimal repair bandwidth or optimal rebuilding access for the systematic nodes only can be converted into one with the corresponding optimality property for all nodes. In fact, the second application can also be applied to any binary MDS array code with an efficient but not optimal repair for the systematic nodes, such as the MDS array codes in [2, 34], to enable optimal repair of all the parity nodes while keeping the repair efficiency of the systematic nodes. The proposed transformation is also applicable to nonbinary MDS array codes and RS codes, but the performance is no better than those in [31, 32, 33].

We note that another transformation for binary MDS array codes was proposed recently in [35], which has a similar flavor as the one proposed in this paper. The main technique used in the transformation in [35] is similar to that in [31], i.e., by operating on multiple instances of a base code and pairing data. The difference is that in [35] the data stored in each node is regarded as a polynomial of degree less than the sub-packetization level and thus the pairing step is carried out in a polynomial ring. Compared with the one in [35], the generic transformation proposed in this paper has several advantages, such as providing a uniform procedure, having a wide potential for applications, and enabling low complexity. Detailed comparisons will be carried out in Section IV-C. Very recently, as a variant of the transformation in [35], another transformation that was tailored for Zigzag-decodable reconstruction (ZDR) codes was proposed in [38]. ZDR codes are not MDS, hence they are left out of consideration in this paper. Although it was mentioned that the recent transformation in [38] can also be applied to binary MDS array codes, however, it is only applicable to those whose repair matrices satisfy a strict condition (cf. [38, Theorem 7]), which greatly limits its application and makes it inferior to the transformation in [38] in terms of the range of the application to binary MDS array codes.

The remainder of the paper is organized as follows. Section II gives some necessary preliminaries. The generic transformation is given in Section III, followed by the proofs of the asserted properties. Two specific applications of the transformation as well as comparisons are discussed in Section IV. Finally, Section V draws the conclusion.

II Preliminaries

An $[N,K,D]$ (linear) code over $\mathbf{F}_{q}$ is a subspace of $\mathbf{F}_{q}^{N}$ , where the minimum Hamming distance $D$ is the smallest weight of the nonzero codewords or nonzero vectors in the subspace. The Singleton bound implies that $D$ satisfies $D\leq N-K+1$ for any linear code, linear codes with $D=N-K+1$ are called MDS codes and denoted as $[N,K]$ MDS codes as the minimum distance is clear [36]. An $[n,k]$ array code $\mathcal{C}$ over $\mathbf{F}_{q}^{b}$ is said to be $\mathbf{F}_{q}$ -linear if $\mathcal{C}$ is a vector space over $\mathbf{F}_{q}$ of dimension $nb$ , and is MDS if the Hamming distance equals $n-k+1$ , where the Hamming distance is measured respect to the symbols in $\mathbf{F}_{q}^{b}$ . An array code can be specified by either its parity-check matrix $H$ of size $(n-k)b\times nb$ or its generator matrix $G$ of size $kb\times nb$ , both over $\mathbf{F}_{q}$ . The code is systematic if $H$ contains $\mathbf{I}_{(n-k)b}$ or $G$ contains $\mathbf{I}_{kb}$ , where $\mathbf{I}_{m}$ denotes the identity matrix of order $m$ [28].

In this paper, we only focus on binary MDS array codes, i.e., $\mathbf{F}_{2}$ -linear MDS code over $\mathbf{F}_{2}^{\alpha}$ for some integer $\alpha$ , which is called the sub-packetization level, thus all the scalars, vectors, and matrices are binary. First of all, we fix some notations, which will be used throughout this paper.

•

“ $+$ ” is just the XOR operation in “ $a+b$ ” for $a,b\in\mathbf{F}_{2}$ .

•

A lowercase letter in bold (for example, $\mathbf{a}$ ) denotes a binary column vector.

•

$[i,j)$ denotes the set $\{i,i+1,\cdots,j-1\}$ for any two integers $i<j$ .

Assume the original file to be stored is comprised of $M=k\alpha$ symbols over $\mathbf{F}_{2}$ . Encoding the original file by an $[n=k+r,k]$ MDS array code over $\mathbf{F}_{2}^{\alpha}$ , $n$ column vectors $\mathbf{f}_{0},\mathbf{f}_{1},\cdots,\mathbf{f}_{n-1}$ of length $\alpha$ are obtained. As common practice (including in the seminal work [6]), we assume that the data in these $n$ column vectors is dispersed across $n$ storage nodes, with each node storing one column vector. If the code is systematic, then the $k$ nodes storing the original file are named the systematic nodes while the remaining nodes are referred to as the parity nodes.

An $[n,k]$ MDS array code possesses the MDS property that the original file can be reconstructed by contacting any $k$ out of the $n$ nodes, and is preferable to have optimal repair bandwidth, i.e., a failed node can be regenerated by downloading $\alpha/r$ symbols from each surviving node [6]. Generally, the data downloaded from node $j$ to repair node $i$ can be represented by $S_{i,j}\mathbf{f}_{j}$ , where $S_{i,j}$ is an $\alpha\times\alpha$ matrix with its rank indicating the amount of data that should be downloaded, and $S_{i,j}$ is usually referred to as repair matrix. Besides the repair bandwidth, the rebuilding access (also known as repair access in [35]) should also be optimized. Formally, rebuilding access is the amount of data that needs to be accessed (or read) to repair a failed node, which is of course no less than the repair bandwidth. It would be preferable for an MDS array code to have optimal rebuilding access, which can be achieved if only $\alpha/r$ symbols are accessed at each surviving node [25], i.e., there are exactly $\alpha/r$ nonzero columns in $S_{i,j}$ , where $i,j\in[0,n)$ with $i\neq j$ . This appealing property reinforces the repair bandwidth requirement, and can reduce the disk I/O overhead during the repair process.

III A Generic Transformation for Binary MDS Array Codes

In this section, we present a generic transformation that can convert any $[n,k]$ binary MDS array code with even sub-packetization level111Note that most known binary MDS array codes in the literature have even sub-packetization level [2, 3, 4, 5, 35, 12]. Even when the sub-packetization level of a binary MDS array code is odd, one can combine two instances of such a code in advance so that the sub-packetization level of the resultant code is even. into a new one, which allows an arbitrary set of $r=n-k$ nodes to have both optimal repair bandwidth and optimal rebuilding access. Choosing some $[n,k]$ binary MDS array code as the base code, the $r$ nodes which we wish to endow with optimal rebuilding access are named the target nodes, while the remaining $k$ nodes are called the remainder nodes. W.L.O.G., we always assume that the last $r$ nodes are the target nodes unless otherwise stated. To simplify the notation in what follows, a target node (resp. remainder node) is shortly denoted by TN (resp. RN). Before describing the generic transformation, we first present an example to illustrate the main idea behind it.

III-A An Example $[9,6]$ MDS array Code

Let $\mathcal{C}_{1}$ be a $[9,6]$ MDS array code over $\mathbf{F}_{2}^{\alpha^{\prime}}$ , where $\alpha^{\prime}$ is even. We generate three codewords of the given code. To this end, let $(\mathbf{f}_{0}^{(l)},\mathbf{f}_{1}^{(l)},\cdots,\mathbf{f}_{5}^{(l)},\mathbf{g}_{0}^{(l)},\mathbf{g}_{1}^{(l)},\mathbf{g}_{2}^{(l)})$ be a codeword/instance of the base code $\mathcal{C}_{1}$ , where $l\in[0,3)$ and the nodes storing $\mathbf{g}_{0}^{(l)},\mathbf{g}_{1}^{(l)},\mathbf{g}_{2}^{(l)}$ are designated as the target nodes. That is, the original file is first encoded into three codewords of the base code. For each code symbol, we divide it into two equal parts. That is, rewrite $\mathbf{f}_{i}^{(l)}$ ( $i\in[0,5)$ ) and $\mathbf{g}_{j}^{(l)}$ ( $j\in[0,3)$ ) as

[TABLE]

By applying the generic transformation, a new $[9,6]$ MDS array code with sub-packetization level $\alpha=3\alpha^{\prime}$ is obtained, which is shown in Table I. The new code has optimal rebuilding access for the $3$ target nodes, which are denoted as

[TABLE]

where

[TABLE]

for $i,l\in[0,3)$ .

Reconstruction: W.L.O.G., let us consider reconstructing the original file by contacting nodes $2$ to $7$ (i.e., $\mathbf{f}_{2},\ldots\mathbf{f}_{5}$ , $\mathbf{h}_{0}$ , and $\mathbf{h}_{1}$ ). According to Table I, we can get the components of $\mathbf{g}_{0}^{(1)}$ and $\mathbf{g}_{1}^{(0)}$ in the following manner.

[TABLE]

In conjunction with the remaining data in rows 1 and 2 at nodes $2$ to $7$ , we now obtain

[TABLE]

from which $(\mathbf{f}_{0}^{(0)},\ldots,\mathbf{f}_{5}^{(0)})$ and $(\mathbf{f}_{0}^{(1)},\ldots,\mathbf{f}_{5}^{(1)})$ can be reconstructed, respectively, according to the MDS property of the base code. Then $\mathbf{g}_{2}^{(0)}$ and $\mathbf{g}_{2}^{(1)}$ can be computed from these available data, and together with $\mathbf{h}_{0}^{(2)}$ and $\mathbf{h}_{1}^{(2)}$ , we can obtain $\mathbf{g}_{0}^{(2)}$ and $\mathbf{g}_{1}^{(2)}$ since

[TABLE]

Finally, in conjunction with the remaining data in the last row at nodes $2$ to $5$ , we now obtain

[TABLE]

from which $(\mathbf{f}_{0}^{(2)},\ldots,\mathbf{f}_{5}^{(2)})$ can be reconstructed. The original file is therefore reconstructed according to the above analysis.

Optimal rebuilding access for the target nodes: W.L.O.G., consider repairing target node [math], and we download the following data

[TABLE]

i.e., the data in row 1 of Table I. Clearly, $\mathbf{g}_{0}^{(0)}$ can be obtained from $\mathbf{f}_{0}^{(0)},\ldots,\mathbf{f}_{5}^{(0)}$ . To compute

[TABLE]

that was stored at target node [math], observe firstly that $\mathbf{g}_{1}^{(0)}=\left(\begin{array}[]{c}\mathbf{g}_{1,0}^{(0)}\\ \mathbf{g}_{1,1}^{(0)}\\ \end{array}\right)$ can be computed with $\mathbf{f}_{0}^{(0)},\mathbf{f}_{1}^{(0)},\ldots,\mathbf{f}_{5}^{(0)}$ , then we can regenerate $\mathbf{g}_{0}^{(1)}=\left(\begin{array}[]{c}\mathbf{g}_{0,0}^{(1)}\\ \mathbf{g}_{0,1}^{(1)}\\ \end{array}\right)$ from the downloaded data $\mathbf{h}_{1}^{(0)}=\mathbf{g}_{1}^{(0)}+\mathbf{g}_{0}^{(1)}$ , and obtain $\mathbf{h}_{0}^{(1)}$ subsequently. The other piece of coded data $\mathbf{h}_{0}^{(2)}$ stored at target node [math] can be similarly regenerated. Thus target node [math] can indeed be optimally repaired and has optimal rebuilding access according to (1).

III-B A Key Pairing

In this subsection, we introduce a pairing technique of two column vectors and analyze its properties, which will be crucial for the generic transformation.

Let $N$ denote an even constant from now on. For any two column vectors $\mathbf{a}{[i]}$ and $\mathbf{b}{[i]}$ of length $N$ , we divide them into two equal parts, which can be represented as

[TABLE]

Then we define a linear operation $\boxplus$ between two column vectors $\mathbf{a}{[i]}$ and $\mathbf{b}{[i]}$ of length $N$ as

[TABLE]

which takes $\frac{3N}{2}$ XORs, where $\mathbf{0}_{N/2}$ denotes the zero matrix of order $N/2$ .

For any two column vectors $\mathbf{a}$ and $\mathbf{b}$ of length $\delta N$ , where $\delta\geq 1$ , we divide them into $\delta$ segments, i.e., rewrite $\mathbf{a}$ and $\mathbf{b}$ as

[TABLE]

where $\mathbf{a}{[i]}$ and $\mathbf{b}{[i]}$ are column vectors of length $N$ and are named the $i$ th segments of $\mathbf{a}$ and $\mathbf{b}$ for $i\in[0,\delta)$ , respectively. Based on (6), we further define a linear operation $\boxplus_{N}$ between two column vectors $\mathbf{a}$ and $\mathbf{b}$ of length $\delta N$ as

[TABLE]

i.e., performing the linear operation $\boxplus$ defined in (6) on the each segments of $\mathbf{a}$ and $\mathbf{b}$ , which takes $\frac{3\delta N}{2}$ XORs. Then, the following fact is obvious.

Fact 1.

For any two column vectors $\mathbf{a}$ and $\mathbf{b}$ of length $\delta N$ with $\delta\geq 1$ , one can get

(i)

$\mathbf{a}[i]$ * and $\mathbf{b}[i]$ from $\mathbf{a}[i]+\mathbf{b}[i]$ and $\mathbf{a}[i]\boxplus_{N}\mathbf{b}[i]$ by $2N$ XORs,*

(ii)

$\mathbf{a}[i]$ * from $\mathbf{b}[i]$ and $\mathbf{a}[i]\boxplus_{N}\mathbf{b}[i]$ (or $\mathbf{b}[i]\boxplus_{N}\mathbf{a}[i]$ or $\mathbf{a}[i]+\mathbf{b}[i]$ ) by at most $\frac{3N}{2}$ XORs,*

(iii)

$S\cdot\mathbf{a}[i]$ * and $S\cdot\mathbf{b}[i]$ from $S(\mathbf{a}[i]+\mathbf{b}[i])$ and $S(\mathbf{a}[i]\boxplus_{N}\mathbf{b}[i])$ by $4L$ XORs,*

for all $i\in[0,\delta)$ and $2L\times N$ matrix $S=\left(\begin{array}[]{cc}S^{\prime}&\\ &S^{\prime}\end{array}\right)$ with $L$ being any positive integer.

Remark 1.

The operation in (6) can be viewed as multiple basic linear operations between two binary column vectors of length $2$ . This basic linear operation can also be interpreted as an operation between two elements in $\mathbf{F}_{2^{2}}$ or two polynomials in $\mathbf{F}_{2}[x]$ with degree less than $2$ . Such equivalent interpretations also hold for the operation in (23).

III-C A Generic Transformation

In this subsection, we propose the generic transformation, which employs a known $[n=k+r,k]$ binary MDS array code $\mathcal{C}_{1}$ with sub-packetization level $\alpha^{\prime}=\delta N$ as the base code, where $\delta\geq 1$ and $r\geq 2$ . The transformation is then carried out through the following three steps.

Step 1: GENERATING $r$ instances of the base code $\mathcal{C}_{1}$

Assume generating $r$ instances of the code $\mathcal{C}_{1}$ to obtain an intermediate MDS array code $\mathcal{C}_{2}$ with sub-packetization level $\alpha=r\alpha^{\prime}$ , and denote by $\mathbf{f}_{i}^{(l)}$ and $\mathbf{g}_{j}^{(l)}$ the data stored at remainder node $i$ and target node $j$ of the $l$ -th instance of $\mathcal{C}_{1}$ , respectively, where $i\in[0,k)$ and $l,j\in[0,r)$ .

Step 2: PERMUTING the data in the target nodes of $\mathcal{C}_{2}$

Keeping the remainder nodes of $\mathcal{C}_{2}$ intact, we construct another intermediate MDS array code $\mathcal{C}_{3}$ by permuting the data in the target nodes of $\mathcal{C}_{2}$ , which is illustrated as follows. Denote by $\mathbf{h}_{j}$ the data stored at target node $j$ of code $\mathcal{C}_{3}$ . Let us write $\mathbf{h}_{j}$ as

[TABLE]

for convenience, where $\mathbf{h}_{j}^{(l)}$ is defined as

[TABLE]

and $\pi_{0},\pi_{1},\cdots,\pi_{r-1}$ are some $r$ specific permutations on $[0,r)$ , more details of the requirements of the permutations will be given in Theorem 3.

Step 3: PAIRING the data in the target nodes of $\mathcal{C}_{3}$

By modifying only the data at the target nodes of the code $\mathcal{C}_{3}$ while keeping its remainder nodes intact, we construct the desired storage code $\mathcal{C}_{4}$ as follows. Denote by $\mathbf{h^{\prime}}_{j}$ the data stored at target node $j$ of code $\mathcal{C}_{4}$ . Let us write $\mathbf{h^{\prime}}_{j}$ as

[TABLE]

for convenience, where $\mathbf{h^{\prime}}_{j}^{(l)}$ , $j,l\in[0,r)$ , are defined by

[TABLE]

and the linear operation $\boxplus_{N}$ is defined in (23). It is easy to see that this step takes a total of $\frac{5r(r-1)}{4}\alpha^{\prime}$ XORs. The new code $\mathcal{C}_{4}$ is depicted in Table II.

In the following, we show that the new code $\mathcal{C}_{4}$ maintains the MDS property of the base code. The proof is similar to that in [31], nevertheless, we include it for completeness.

Theorem 1.

Code $\mathcal{C}_{4}$ has the MDS property. More specifically, the reconstruction process of the code $\mathcal{C}_{4}$ requires invoking the reconstruction process of the base code $r$ times and at most $\frac{1}{2}(3rt-t^{2}-2t)\alpha^{\prime}$ additional XORs, where $t$ denotes the number of target nodes that are connected during the reconstruction, for $0\leq t\leq\min\{r,k\}$ .

Proof.

The code $\mathcal{C}_{4}$ has the MDS property if the original file (or equivalently, $\mathbf{f}_{i}^{(l)}$ , $i\in[0,k)$ and $l\in[0,r)$ ) can be reconstructed by connecting to any $k$ out of the $n$ nodes. The reconstruction is discussed in the following two cases.

(i) When connecting to $k$ remainder nodes: the original file can be reconstructed according to the MDS property of the base code.

(ii) Suppose $k-t$ remainder nodes and $t$ target nodes are connected, where $1\leq t\leq\min\{r,k\}$ : let $I=\{i_{0},i_{1},\cdots,i_{t-1}\}$ denote the set of the indices of the $t$ remainder nodes that are not connected and $J=\{j_{0},j_{1},\cdots,j_{t-1}\}$ be the set of the indices of the connected target nodes, where $0\leq i_{0}<\cdots<i_{t-1}<k$ , $0\leq j_{0}<\cdots<j_{t-1}<r$ . Denote $\{j_{t},\cdots,j_{r-1}\}=[0,r)\backslash J$ . For example, in the example in Section III-A, $t=2$ , $I=\{0,1\}$ , $J=\{0,1\}$ , and $j_{2}=2$ .

Firstly, from the $t$ connected target nodes, the data in Table III is available, from which in conjunction with Fact 1-(i), we can get $\mathbf{h}_{u}^{(l)}$ ( $l,u\in J$ ) by $t(t-1)\alpha^{\prime}$ XORs.

Secondly, for each $l\in J$ , by $\mathbf{h}_{u}^{(l)}$ ( $u\in J$ ) (at the target nodes) and $\mathbf{f}_{i}^{(l)}$ ( $i\in[0,k)\backslash I$ ) (at the $k-t$ remainder nodes of code $\mathcal{C}_{4}$ connected), $\mathbf{h}_{u}^{(l)}$ , $u\in[0,r)\backslash J$ can be reconstructed by (24) and the fact that $\mathcal{C}_{1}$ is MDS.

Thirdly, for the data in Table IV, by Fact 1-(ii), one can obtain $\mathbf{h}_{u}^{(l)}$ ( $l\in[0,r)\backslash J$ , $u\in J$ ) after cancelling $\mathbf{h}_{l}^{(u)}$ ( $u\in J$ , $l\in[0,r)\backslash J$ ) (marked with dash underline) by at most $\frac{3}{2}t(r-t)\alpha^{\prime}$ XORs. Now, $\mathbf{h}_{u}^{(l)}$ , i.e., $\mathbf{g}_{\pi_{l}(u)}^{(l)}$ is available for all $l\in[0,r)$ and $u\in J$ .

Finally, together with $\mathbf{f}_{i}^{(l)}$ , $i\in[0,k)\backslash I$ , for each $l\in[0,r)$ , the remaining data $\mathbf{f}_{i_{0}}^{(l)},\cdots,\mathbf{f}_{i_{t-1}}^{(l)}$ can be reconstructed according to the MDS property of $\mathcal{C}_{1}$ .

From the above analysis, we see that reconstructing the original file from $k-t$ remainder nodes and $t$ target nodes requires invoking the reconstruction process of the base code $r$ times and at most $\frac{\alpha^{\prime}}{2}(3rt-t^{2}-2t)$ additional XORs, where $0\leq t\leq\min\{r,k\}$ .

∎

Next, we focus on the repair of the target nodes of code $\mathcal{C}_{4}$ .

Theorem 2.

The target nodes of code $\mathcal{C}_{4}$ have optimal rebuilding access. More specifically, the repair of a target node requires invoking a reconstruction (or encoding) process of the base code and an extra of $\frac{5}{2}(r-1)\alpha^{\prime}$ XORs.

Proof.

For any given $j\in[0,r)$ , in the following, we prove that target node $j$ can be repaired by accessing and downloading $\mathbf{h^{\prime}}^{(j)}_{l}$ , $l\in[0,r)\backslash\{j\}$ , and $\mathbf{f}^{(j)}_{i}$ , $i\in[0,k)$ .

Firstly, one can compute $\mathbf{g}^{(j)}_{s}$ , $s\in[0,r)$ from $\mathbf{f}^{(j)}_{i}$ , $i\in[0,k)$ according to the MDS property of the base code, and then obtain $\mathbf{h}^{(j)}_{s}$ , $s\in[0,r)$ by (24). Secondly, for any $0\leq l\neq j<r$ , according to the downloaded data $\mathbf{h^{\prime}}^{(j)}_{l}=\mathbf{h}^{(j)}_{l}\boxplus_{N}\mathbf{h}^{(l)}_{j}$ (if $l<j$ ) or $\mathbf{h^{\prime}}^{(j)}_{l}=\mathbf{h}^{(j)}_{l}+\mathbf{h}^{(l)}_{j}$ (if $l>j$ ) and Fact 1-(ii), $\mathbf{h}^{(l)}_{j}$ is available after cancelling $\mathbf{h}^{(j)}_{l}$ from $\mathbf{h^{\prime}}^{(j)}_{l}$ , and thus one can get $\mathbf{h^{\prime}}^{(l)}_{j}=\mathbf{h}^{(l)}_{j}+\mathbf{h}^{(j)}_{l}$ (if $l<j$ ) or $\mathbf{h^{\prime}}^{(l)}_{j}=\mathbf{h}^{(l)}_{j}\boxplus_{N}\mathbf{h}^{(j)}_{l}$ (if $l>j$ ). Now target node $j$ is regenerated by noting that $\mathbf{h^{\prime}}^{(j)}_{j}=\mathbf{h}^{(j)}_{j}$ . The repair of target node $j$ requires invoking a reconstruction (or encoding) process of the base code and an additional $\frac{5}{2}(r-1)\alpha^{\prime}$ XORs according to Fact 1. ∎

Finally, we analyze the repair of the remainder nodes of code $\mathcal{C}_{4}$ .

Theorem 3.

For each $i\in[0,k)$ , remainder node $i$ of the $[n,k]$ MDS array code $\mathcal{C}_{4}$ maintains the same normalized repair bandwidth and rebuilding access as those of the base code if the repair strategy for remainder node $i$ of the base code is naive, or during the repair process of node $i$ of the base code, either the following condition (i) or (ii) holds:

(i)

R1 holds for $j\in[0,r)$ , and either R2 or R3 holds;

(ii)

$r=2$ , R1 holds for $j=0$ or $j=1$ , and R2 holds,

where R1-R3 are defined as follows.

R1.

The repair matrix $S_{i,k+j}$ is a block diagonal matrix of the form

[TABLE]

where $S_{i,k+j,m}$ is an $\frac{N}{2}\times\frac{N}{2}$ matrix (can also be a zero matrix) and is repeated twice in the above matrix for every $m\in[0,\delta)$ ;

R2.

$\pi_{l}(j)=\pi_{j}(l)$ * for $j,l\in[0,r)$ ;*

R3.

$S_{i,k,m}=S_{i,k+j,m}$ * for all $j\in[1,r)$ and $m\in[0,\delta)$ .*

In addition, repairing remainder node $i$ of the code $\mathcal{C}_{4}$ requires invoking the repair strategy of the base code $r$ times, and an extra of $(r-1)\alpha^{\prime}$ XORs if the repair strategy for remainder node $i$ of the base code is non-naive.

Proof.

If the repair strategy for remainder node $i$ of the base code is naive, then the same holds for the new code $\mathcal{C}_{4}$ . Herein we only now focus on the non-naive case.

Suppose condition (i) holds, then for any $i\in[0,k)$ , $l\in[0,r)$ , according to the repair strategy of the base code and R1, we have that $\mathbf{f}_{i}^{(l)}$ can be repaired by downloading $S_{i,s}\mathbf{f}_{s}^{(l)}$ , $s\in[0,k)\backslash\{i\}$ , and

[TABLE]

from the surviving nodes, where $m\in[0,\delta)$ .

If R2 or R3 holds, then we have

[TABLE]

for all $l,j\in[0,r)$ with $l\neq j$ and $m\in[0,\delta)$ . Then, the repair process for remainder node $i$ of code $\mathcal{C}_{4}$ can be proceeded as follows:

(a)

Download $S_{i,s}\mathbf{f}_{s}^{(l)}$ and

[TABLE]

for all $s\in[0,k)\backslash\{i\}$ , $j,l\in[0,r)$ , and $m\in[0,\delta)$ ,

(b)

For all $l,j\in[0,r)$ with $l\neq j$ , by Fact 1, (30), and from

[TABLE]

and

[TABLE]

we can get

[TABLE]

and

[TABLE]

by $2N$ XORs according to Fact 1-(ii). By (24), and together with the data

[TABLE]

obtained in the previous step, the data

[TABLE]

i.e., $S_{i,k+j}\mathbf{g}_{j}^{(l)}$ , is available now for all $l,j\in[0,r)$ by a total of $r(r-1)N$ XORs.

(c)

From the data obtained in the previous two steps, the repair procedure of the base MDS code can be invoked to regenerate $\mathbf{f}_{i}^{(l)}$ for $l\in[0,r)$ .

Therefore, repairing remainder node $i$ requires invoking the repair strategy of the base code $r$ times and an extra of $(r-1)\alpha^{\prime}$ XORs.

If condition (ii) holds, the proof proceeds in the same fashion, thus we omit it. ∎

Note that if condition (ii) of Theorem 3 does not hold, then condition (i) has to be fulfilled to apply Theorem 3. In this case, for simplicity of verification, one could only check R1 when applying Theorem 3, as Requirement R2 of Theorem 3 can be easily satisfied, e.g., set $\pi_{l}(j)=(l+j)\bmod r$ , for $l,j\in[0,r)$ . Nevertheless, this does not mean that R3 of Theorem 3 is unnecessary. If R1 and R3 hold, then there is no restriction on the permutations in Step 2 as condition (i) of Theorem 3 is now satisfied naturally. Therefore, the permutations can be arbitrary including the identity permutation, i.e., Step 2 of the generic transformation can be omitted, which can simplify the transformation.

Remark 2.

Note that for a binary MDS array code with sub-packetization level $\alpha^{\prime}$ , there may be several pairs of $(\delta,N)$ such that $\alpha^{\prime}=\delta N$ and $N$ is even. To apply the generic transformation, one should choose an appropriate pair of $(\delta,N)$ so that either condition (i) or condition (ii) of Theorem 3 can be satisfied.

To this end, we give a concluding result of this subsection according to Theorems 1-3.

Theorem 4.

For an $[n,k]$ binary MDS array code with sub-packetization level $\alpha^{\prime}=\delta N$ , by applying the generic transformation in Section III-C, a new $[n,k]$ binary MDS array code with sub-packetization level $\alpha=r\alpha^{\prime}$ can be obtained such that

•

Any arbitrarily chosen $n-k$ nodes have optimal rebuilding access.

•

The remaining nodes can maintain the same normalized repair bandwidth and rebuilding access as those of the base code if the repair strategy for the remainder nodes of the base code is naive, or if either condition (i) or (ii) in Theorem 3 holds.

III-D An Alternative Pairing Technique for Step 3 - Target Nodes Unchanged

Note that the code obtained by the generic transformation in Section III-C is no longer of the systematic form if some $r$ systematic nodes of a systematic base code are chosen to be the target nodes. Although it is a well-known fact that a linear code in non-systematic form can always be converted into a systematic one, such a conversion is not trivial/easy in general since normally one needs to solve a linear equation system. To this end, here we provide a substitute pairing operation that can maintain the systematic form of the code without performing further conversion, which can also avoid the complexity caused by the conversion.

W.L.O.G., the last $r$ nodes are chosen as the target nodes while the data modification is carried out at the first $r$ nodes. Note that $\mathbf{f}_{0}^{(l)},\cdots,\mathbf{f}_{r-1}^{(l)}$ can be represented by $\mathbf{f}_{r}^{(l)},\cdots,\mathbf{f}_{k-1}^{(l)},\mathbf{g}_{0}^{(l)},\cdots$ , $\mathbf{g}_{r-1}^{(l)}$ for any $l\in[0,r)$ by the MDS property of the base code $\mathcal{C}_{1}$ . That is,

[TABLE]

where $A_{j,0},\cdots,A_{j,k-1}$ are some nonsingular matrices of order $N$ . Then, define a new storage code $\mathcal{C}_{4}^{\prime}$ (see Table V) as

[TABLE]

where $\mathbf{v}_{t}^{(l)}$ is defined in (31).

Note from (31) that for $t,l\in[0,r)$ with $t>l$ , we have

[TABLE]

which together with (23) we further have $\mathbf{v}_{t}^{(l)}+\mathbf{v}_{l}^{(t)}=\mathbf{h}_{t}^{(l)}$ and $\mathbf{v}_{l}^{(t)}\boxplus_{N}\mathbf{v}_{t}^{(l)}=\mathbf{h}_{l}^{(t)}$ . That is, the new code $\mathcal{C}_{4}^{\prime}$ can be obtained by applying Step 3 to the code $\mathcal{C}_{3}^{\prime}$ (cf. Table VI), where

[TABLE]

is an instance of $\mathcal{C}_{1}$ by (32).

Similarly to [31], we immediately have the following result.

Theorem 5.

The code $\mathcal{C}_{4}^{\prime}$ has the MDS property and the same repair property as that of the code $\mathcal{C}_{4}$ .

IV Applications and comparisons

In this section, based on the transformation in Section III, we propose two specific applications. The first application is recursively applying the transformation multiple times to an arbitrary binary MDS array code to obtain a new one with optimal rebuilding access for all nodes. The second application is applying the transformation one time to an arbitrary binary MDS array codes with optimal repair bandwidth/optimal rebuilding access for all systematic nodes to obtain a new one with the corresponding optimality property for all nodes. Finally, we provide extensive comparisons between the transformation proposed in this paper and the one in [35].

IV-A Application From an Arbitrary Binary MDS Array Code

Choosing any $[n,k]$ binary MDS array code with even sub-packetization level $\alpha^{\prime}$ as the base code $\mathcal{Q}_{0}$ , for example, one can choose the EVENODD code [2] or its generalizations in [3, 4], or the STAR code [5] as the base code. We can build a binary MDS array code with optimal rebuilding access for all nodes through Algorithm 1.

According to Theorems 1-3, Algorithm 1 eventually gives a binary MDS array code $\mathcal{Q}_{m}$ with optimal rebuilding access, where the sub-packetization level is $r^{\lceil n/r\rceil}\alpha^{\prime}$ . Specifically, applying Algorithm 1 to the $[n=k+2,k]$ EVENODD code with sub-packetization level $\alpha^{\prime}=k-1$ in [2], where $k$ is an odd prime, we can obtain an $[n=k+2,k]$ MDS array code with optimal rebuilding access for all nodes, where the sub-packetization level is $(k-1)2^{\frac{n+1}{2}}$ , which is $k-1$ times the lower bound of the minimal required sub-packetization level derived in [9]. However, the lower bound in [9] is not sensitive to the underlying finite field.

Strictly speaking, to recursively apply Theorem 3, one needs to verify that requirement R1 of Theorem 3 is satisfied during each round of the transformation in Algorithm 1. Indeed, it can be checked by induction as follows.

Induction hypothesis: For $t\in[1,m-1)$ , the first $tr$ nodes of the code $\mathcal{Q}_{t}$ can be optimally repaired and the $r^{t}\alpha^{\prime}\times r^{t}\alpha^{\prime}$ repair matrix $S_{i,j}^{(t)}$ of node $i$ ( $i\in[0,tr)$ ) is of the form

[TABLE]

and

[TABLE]

for all $j\in[0,n)\backslash\{i\}$ and $w\in[0,r^{t})$ with $(w_{t-1},\cdots,w_{1},w_{0})$ denoting its $r$ -ary expansion, where $l$ is a nonnegative integer such that $lr\leq i<(l+1)r$ .

In the $t$ -th round, the MDS array code $\mathcal{Q}_{t}$ is set as the base code, and nodes

[TABLE]

are designated as target nodes. Clearly, the repair matrices of the first $\min\{tr,k-r\}$ nodes of the base code $\mathcal{Q}_{t}$ satisfy the requirement R1 of Theorem 3 by (34) and (37). Further, the induction hypothesis is true for all the $r^{t+1}\alpha^{\prime}\times r^{t+1}\alpha^{\prime}$ repair matrices $S_{i,j}^{(t+1)}$ of the first $\min\{(t+1)r,k\}$ nodes of the MDS array code $\mathcal{Q}_{t+1}$ since

[TABLE]

for $i\in[0,\min\{tr,k-r\}$ ), $j\in[0,n)\backslash\{i\}$ , and

[TABLE]

for $i\in[\min\{tr,k-r\},\min\{(t+1)r,k\})$ and $j\in[0,n)\backslash\{i\}$ with $S_{i,j,w}^{(t+1)}$ being defined in (33) according to Theorem 2.

In what follows, we give an example of the application.

Example 1.

For a $[5,3]$ EVENODD code $\mathcal{Q}_{0}$ , which is binary and has a sub-packetization level of $2$ , the structure of $\mathcal{Q}_{0}$ can be depicted as in Table VII [2].

In the following, through three rounds of transformations according to Algorithm 1, we convert $\mathcal{Q}_{0}$ into an MDS array code with optimal rebuilding access for all nodes. We obtain codes $\mathcal{Q}_{1}$ , $\mathcal{Q}_{2}$ and $\mathcal{Q}_{3}$ , where $\mathcal{Q}_{1}$ and $\mathcal{Q}_{2}$ are shown in Tables VIII and IX, respectively, while the representation of $\mathcal{Q}_{3}$ is omitted due to the limited space. For simplicity, all the permutations in Step 2 of each round are chosen to be the identity permutation, as R3 of Theorem 3 is obviously satisfied for the non-naively repaired remainder nodes of the intermediate code $\mathcal{Q}_{i}$ ( $i\in[1,m)$ ).

Note that during the first two rounds of transformations, the alternative pairing technique in Section III-D is employed in Step 3, i.e., data modification is carried out at the parity nodes. For example, when applying the generic transformation to the code $\mathcal{Q}_{0}$ , after Steps 1 and 2, we get an intermediate code $\mathcal{Q}^{\prime}_{0}$ as in Table X, while in Step 3, $b_{0},~{}b_{1},~{}a_{2}$ and $a_{3}$ in the parity nodes are respectively replaced by $a_{2}+a_{3}+b_{0}+b_{1},~{}a_{2}+b_{0},~{}a_{2}+a_{3}+b_{1},~{}a_{2}+b_{0}+b_{1}$ according to (31), which leads to the code $\mathcal{Q}_{1}$ in Table VIII.

It is seen that $\mathcal{Q}_{2}$ maintains the MDS property. Furthermore, the first four nodes can be respectively optimally repaired by accessing and downloading symbols in rows $\{1,2,5,6\}$ , $\{3,4,7,8\}$ , $\{1,2,3,4\}$ , $\{5,6,7,8\}$ of Table IX from all the surviving nodes.

IV-B Application From Binary MDS array Codes With Optimal Repair for Systematic Nodes

In the literature, there are some binary MDS array codes with optimal repair bandwidth for all the systematic nodes but not all the parity nodes. For example, the $[n=k+2,k]$ MDR code in [10] and the two modified versions [11]. In this subsection, we propose the second application of the generic transformation by presenting Algorithm 2, which can build a binary MDS array code with optimal repair bandwidth for all nodes from one with optimal repair bandwidth for all the systematic nodes but not all the parity nodes.

Note that in Algorithm 2, if neither condition (i) nor condition (ii) of Theorem 3 holds for $\mathcal{C}_{1}$ , then for $i\in[0,~{}k)$ and $j\in[0,~{}r)$ , the repair matrix $S_{i,k+j}^{\mathcal{C}_{2}}$ of systematic node $i$ of $\mathcal{C}_{2}$ has the form

[TABLE]

according to Line 6, thus R1 of Theorem 3 holds for $j\in[0,~{}r)$ , and therefore condition (i) holds, where $S_{i,k+j}^{\mathcal{C}_{1}}$ denotes the repair matrix of systematic node $i$ of $\mathcal{C}_{1}$ . Thus, Algorithm 2 generates an $[n=k+r,k]$ binary MDS array code with optimal repair bandwidth for all nodes by Theorems 1-3, while the sub-packetization level is $2\alpha^{\prime}$ or $4\alpha^{\prime}$ .

In what follows, we give an example by performing the application to the first version of the modified MDR (MDR-1 for short) code in [11], where the two parity nodes of the code are designated as the target nodes. For the $[n=k+2,k]$ MDR-1 code with sub-packetization level $\alpha^{\prime}=2^{k-1}$ in [11], we focus on the general case $k\geq 4$ , and let us first recall the repair strategy of the systematic nodes. For an integer $j$ , where $j\in[0,2^{k-1})$ , let $(j_{k-2},\cdots,j_{1},j_{0})$ be its binary expansion, i.e., $j=j_{0}+2j_{1}+\cdots+2^{k-2}j_{k-2}$ . When repairing node $i$ ( $i\in[0,k)$ ) of the MDR-1 code in [11], one accesses and downloads the $j$ -th element from each surviving node for all $j\in J_{i}$ , where

[TABLE]

Example 2.

For the $[6,4]$ MDR-1 code $\mathcal{C}_{1}$ in [11], which is binary and has a sub-packetization level of $8$ , the structure of $\mathcal{C}_{1}$ can be depicted as in Table XI. By applying the second application to code $\mathcal{C}_{1}$ , we can get a $[6,4]$ binary MDS array code $\mathcal{C}_{3}$ with optimal rebuilding access for all nodes, as shown in Table XII, where $\textbf{f}_{0}^{(l)},\textbf{f}_{1}^{(l)},\textbf{f}_{2}^{(l)},\textbf{f}_{3}^{(l)},\textbf{g}_{0}^{(l)}$ , $\textbf{g}_{1}^{(l)}$ denote an instance of the $[6,4]$ MDR-1 code $\mathcal{C}_{1}$ for $l\in[0,4)$ with $\textbf{g}_{0}^{(l)},\textbf{g}_{1}^{(l)}$ denoting the parity data.

For the code $\mathcal{C}_{3}$ , one can directly verify that the code is MDS. Furthermore, systematic node $i$ ( $i\in[0,4)$ ) is optimally repaired by accessing and downloading the $j$ -th element from the surviving nodes for all $j\in J_{i}$ , where

[TABLE]

Parity nodes $0,1$ can be optimally repaired by accessing and downloading the $j$ -th element from the surviving nodes for all $j\in J_{4}$ and $J_{5}$ , respectively, where

[TABLE]

and

[TABLE]

Remark 3.

In Algorithm 2, we can also choose any binary MDS array code with an efficient but not optimal repair for systematic nodes as the base code, then Algorithm 2 generates another binary MDS array code with optimal rebuilding access for the parity nodes, while the repair efficiency for the systematic nodes is maintained as in the base code. For example, we can apply Algorithm 2 to the $[n=p+2,k=p]$ EVENODD code222The efficient repair of the systematic nodes follows from the strategy in [37]. with sub-packetization level $p-1$ in [2], where $p$ is a prime. Whereas, the transformation in [35] is only applicable to the $[p+2,p]$ EVENODD code with $4\mid(p-1)$ .

IV-C Comparisons Between the Proposed Transformation and the One in [35]

The transformation in [35] also relies on a pairing technique and the data stored in each node is regarded as a polynomial, the two different linear combinations of polynomials $a(x)$ and $b(x)$ are generated as $a(x)+b(x)$ and $a(x)+\delta(x)b(x)$ for some polynomial $\delta(x)$ . Compared to the one in [35], the generic transformation proposed in this paper has the following advantages:

•

The transformation in this paper is uniform. It can be applied to any binary MDS array codes such as the MDS codes in [2, 3, 4, 5, 10, 11, 12] through a unified pairing technique. Whereas the pairing technique in the transformation in [35] is not uniform, i.e., the coefficient polynomial $\delta(x)$ is not uniform but should be chosen case by case such that the statements (which were only proved in general in [35] if the base code is the EVENODD code) still hold for the chosen base code. Whether an appropriate coefficient polynomial $\delta(x)$ can always be found for any existing binary MDS array codes is unknown in [35].

•

The computation complexity of the proposed transformation is very low. To apply the transformation, only multiple linear operations between two binary column vectors of length $2$ is introduced to the base code, whereas, for the transformation in [35], the extra computation cost introduced to the base code is a linear operation between two polynomials in $\mathbf{F}_{2}[x]$ with degree equals to $\alpha^{\prime}-1$ (where $\alpha^{\prime}$ denotes the sub-packetization level of the base code), which needs more XORs and thus results in a higher computation complexity.

For the MDS array codes obtained by the transformation proposed in this work and the one in [35], the reconstruction process and repair process of a target node of the new code are mainly relying on those processes of the base code and some additional XORs. Therefore, it suffices to compare the extra complexity involved, i.e., the numbers of additional XORs that are needed, which mainly depend on the pairing technique involved. The pairing technique is independent of the base code for the transformation proposed in this work, but depends on the base code for the one in [35]. However, in [35], the pairing technique is only given to the EVENODD codes in general in Section II, therefore, we take this pairing technique for a quantified comparison of the extra computational complexity introduced to the transformation proposed in this work and the one in [35] in Table XIII. We see that the transformation proposed in this work indeed has a much lower computational complexity than the one in [35].

•

The proposed transformation has a wider range of applications in the case of $d=n-1$ . It can be applied to any binary MDS array codes, whereas, the applications of the transformation in [35] are limited, whether it can be applied to binary MDS array codes with optimal repair bandwidth such as the codes in [10, 11] is unknown. Particularly, when applying the transformation in [35] one time to the $[n=p+r,k=p]$ EVENODD code to enable optimal rebuilding access for all the parity while keeping the repair efficiency of the systematic nodes, it is valid only if $4\mid(p-1)$ and $r=2$ , while ours does not have such restrictions (see Remark 3).

Table XIV summarizes the above comparison.

Remark 4.

The transformation in [35] also works for binary MDS array codes with $d<n-1$ , however, when repairing a failed node of the resultant code, some of the $d$ helper nodes should be specifically selected. Note that by operating on $d-k+1$ instances of the base code in the first step, the newly proposed transformation can be easily generalized to the case of $d<n-1$ with a restricted selection of helper nodes. Thus, instead of involving the imperfect and complicated case of $d<n-1$ in this paper, we would rather leave the case of $d<n-1$ and the $d$ helper nodes can be arbitrarily chosen in our future work to keep the simplicity of the current paper.

V Conclusion

In this work, we proposed a generic transformation that can be applied to any binary MDS array code to produce new binary MDS array codes with some arbitrarily chosen $r$ nodes having optimal repair bandwidth and optimal rebuilding access. Besides, based on this transformation, we provided two useful applications to yield binary MDS array codes with optimal repair bandwidth/rebuilding access for all nodes. The computation complexity of the new binary MDS array codes obtained by our generic transformation is very low, with only simple XOR operations needed to perform on the base code to complete the encoding, decoding, and repair processes. Extending the transformation to the case of $d<n-1$ with the $d$ helper nodes that can be arbitrarily chosen is part of our ongoing work.

Acknowledgment

The authors would like to thank the Associate Editor Prof. Ido Tal and the three anonymous reviewers for their valuable suggestions and comments, which have greatly improved the presentation and quality of this paper.

Bibliography38

The reference list from the paper itself. Each links out to its DOI / PubMed record.

1[1] I. Reed and G. Solomon, “Polynomial codes over certain finite fields,” J. Soc.Ind. Appl. Math., vol. 8, no. 2, pp. 300–304, Jun. 1960.
2[2] M. Blaum, J. Brady, J. Bruck, and J. Menon, “EVENODD: An efficient scheme for tolerating double disk failures in RAID architectures,” IEEE Trans. on Computers , vol. 44, no. 2, pp. 192–202, 1995.
3[3] M. Blaum, J. Bruck, and A. Vardy, “MDS array codes with independent parity symbols,” IEEE Trans. Inform. Theory, vol. 42, no. 2, pp. 529–542, Mar. 1996.
4[4] M. Blaum, J. Brady, J. Bruck, J. Menon, and A. Vardy, “The EVENODD code and its generalization,” High Performance Mass Storage and Parallel I/O, pp. 187–208, 2001.
5[5] C. Huang and L. Xu, “STAR: an efficient coding scheme for correcting triple storage node failures,” IEEE Trans. Comput., vol. 57, no. 7, pp. 889–901, Jul. 2008.
6[6] A. G. Dimakis, P. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran, “Network coding for distributed storage systems,” IEEE Trans. Inform. Theory, vol. 56, no. 9, pp. 4539–4551, Sep. 2010.
7[7] M. Blaum, P.G. Farell, and H. van Tilborg, “Array codes,” in Handbook of Coding Theory , V. Pless and W. C. Huffman, Eds. Elsevier Science, 1998, vol. II, ch. 22, pp. 1855–1909.
8[8] M. Ye and A. Barg, “Explicit constructions of high-rate MDS array codes with optimal repair bandwidth,” IEEE Trans. Inform. Theory, vol. 63, no. 4, pp. 2001–2014, Apr. 2017.

TL;DR

Contribution

Findings

Abstract

Peer Reviews

Videos

A Generic Transformation for Optimal Node Repair in MDS Array Codes over F2\mathbf{F}_{2}F2​

Abstract

Index Terms:

I Introduction

II Preliminaries

III A Generic Transformation for Binary MDS Array Codes

III-A An Example [9,6][9,6][9,6] MDS array Code

III-B A Key Pairing

Fact 1**.**

Remark 1**.**

III-C A Generic Transformation

Step 1: GENERATING rrr instances of the base code C1\mathcal{C}_{1}C1​

Step 2: PERMUTING the data in the target nodes of C2\mathcal{C}_{2}C2​

Step 3: PAIRING the data in the target nodes of C3\mathcal{C}_{3}C3​

Theorem 1**.**

Proof.

Theorem 2**.**

Proof.

Theorem 3**.**

Proof.

Remark 2**.**

Theorem 4**.**

III-D An Alternative Pairing Technique for Step 3 - Target Nodes Unchanged

Theorem 5**.**

IV Applications and comparisons

IV-A Application From an Arbitrary Binary MDS Array Code

Example 1**.**

IV-B Application From Binary MDS array Codes With Optimal Repair for Systematic Nodes

Example 2**.**

Remark 3**.**

IV-C Comparisons Between the Proposed Transformation and the One in [35]

Remark 4**.**

V Conclusion

Acknowledgment

A Generic Transformation for Optimal Node Repair in MDS Array Codes over $\mathbf{F}_{2}$

III-A An Example $[9,6]$ MDS array Code

Fact 1.

Remark 1.

Step 1: GENERATING $r$ instances of the base code $\mathcal{C}_{1}$

Step 2: PERMUTING the data in the target nodes of $\mathcal{C}_{2}$

Step 3: PAIRING the data in the target nodes of $\mathcal{C}_{3}$

Theorem 1.

Theorem 2.

Theorem 3.

Remark 2.

Theorem 4.

Theorem 5.

Example 1.

Example 2.

Remark 3.

Remark 4.