Collaborative Decoding of Polynomial Codes for Distributed Computation

TL;DR

This paper demonstrates that polynomial codes used in distributed matrix multiplication are interleaved Reed-Solomon codes, enabling collaborative decoding to correct errors and improve numerical stability in distributed computation systems.

Contribution

It reveals the interleaved Reed-Solomon structure of polynomial codes and introduces a collaborative decoding approach for fault-tolerant distributed matrix multiplication.

Findings

Errors can be corrected with probability 1 when t < N-K under Gaussian noise.

Collaborative decoding improves numerical stability as the number of Reed-Solomon codes increases.

The approach enhances fault tolerance in distributed matrix multiplication systems.

Abstract

We show that polynomial codes (and some related codes) used for distributed matrix multiplication are interleaved Reed-Solomon codes and, hence, can be collaboratively decoded. We consider a fault tolerant setup where $t$ worker nodes return erroneous values. For an additive random Gaussian error model, we show that for all $t < N-K$, errors can be corrected with probability 1. Further, numerical results show that in the presence of additive errors, when $L$ Reed-Solomon codes are collaboratively decoded, the numerical stability in recovering the error locator polynomial improves with increasing $L$.

Tables

Table 1. TABLE I : Probability of error for the CPDA, N = 20 ​ K = 12 𝑁 20 𝐾 12 N=20\ K=12 , 12500 trials

$L\backslash t$	1	2	3	4	5	6	7
1	0	0	0	0.0008	-	-	-
2	0	0	0	0	0	-	-
3	0	0	0	0	0	0	-
4	0	0	0	0	0	0	-
5	0	0	0	0	0	0	-
6	0	0	0	0	0	0	-
7	0	0	0	0	0	0	0.0026
8	0	0	0	0	0	0	0.0008
20	0	0	0	0	0	0	0

Equations

$$\mathbf{A}=[{\mathbf{A}}_0{\mathbf{A}}_1\cdots {\mathbf{A}}_{m-1}],\mathbf{B}=[{\mathbf{B}}_0{\mathbf{B}}_1\cdots {\mathbf{B}}_{n-1}].$$

$${\tilde{\mathbf{A}}}_i=\sum _{j=0}^{m-1}{\mathbf{A}}_j{x}_i^{j\alpha },{\tilde{\mathbf{B}}}_i=\sum _{j=0}^{n-1}{\mathbf{B}}_j{x}_i^{j\beta }.$$

$${\tilde{\mathbf{C}}}_i={\tilde{\mathbf{A}}}_i^{\mathsf{T}}{\tilde{\mathbf{B}}}_i=\sum _{j=0}^{m-1}\sum _{k=0}^{n-1}{\mathbf{A}}_j^{\mathsf{T}}{\mathbf{B}}_k{x}_i^{j\alpha +k\beta }.$$

$$\mathbf{P}(x)=\sum _{j=0}^{m-1}\sum _{k=0}^{n-1}{\mathbf{A}}_j^{\mathsf{T}}{\mathbf{B}}_k{x}^{j+km}.$$

$${\tilde{\mathbf{C}}}_i(p,q)\triangleq \sum _{j=0}^{m-1}\sum _{k=0}^{n-1}[{\mathbf{A}}_j^{\mathsf{T}}{\mathbf{B}}_k](p,q)x_i^{j+km}.$$

$$\Lambda (z)\triangleq \prod _{i=1}^t(1-z{\alpha }_{j_i})=1+{\lambda }_{1}z+\cdots +{\lambda }_t{z}^t$$

$${\mathbf{S}}^{(l)}(t)\triangleq \left(\begin{array}{cccc}{S}_0^{(l)}& S_1^{(l)}& \cdots & S_{t-1}^{(l)}\\ S_1^{(l)}& S_2^{(l)}& \cdots & S_t^{(l)}\\ ⋮& ⋮& & ⋮\\ S_{N-K-t-1}^{(l)}& S_{N-K-t}^{(l)}& \cdots & S_{N-K-2}^{(l)}\end{array}\right),{\mathbf{a}}^{(l)}(t)\triangleq \left(\begin{array}{c}-S_t^{(l)}\\ -S_{t+1}^{(l)}\\ ⋮\\ -S_{N-K-1}^{(l)}\end{array}\right)$$

$$\underset{{\mathbf{S}}_L(t)}{\underbrace{\left(\begin{array}{c}{\mathbf{S}}^{(1)}(t)\\ {\mathbf{S}}^{(2)}(t)\\ ⋮\\ {\mathbf{S}}^{(L)}(t)\end{array}\right)}}\underset{\mathbf{\lambda }(t)}{\underbrace{\left(\begin{array}{c}{\lambda }_t\\ {\lambda }_{t-1}\\ ⋮\\ {\lambda }_1\end{array}\right)}}=\underset{{\mathbf{a}}_L(t)}{\underbrace{\left(\begin{array}{c}{\mathbf{a}}^{(1)}(t)\\ {\mathbf{a}}^{(2)}(t)\\ ⋮\\ {\mathbf{a}}^{(L)}(t)\end{array}\right)}}$$

$$\begin{array}{rl}& {\mathcal{E}}_1(t)=\{\mathbf{E}:W_H(\mathbf{E})=t\text{and the MSSR/collaborative }\text{algorithm fails}\},\\ & {\mathcal{E}}_2(t)=\{\mathbf{E}:W_H(\mathbf{E})=t\text{and the MSSR/collaborative }\text{algorithm makes an undetected error}\},\\ & \mathcal{E}(t)=\{\mathbf{E}:W_H(\mathbf{E})=t\}.\end{array}$$

$$P_F(t)=\frac{|{\mathcal{E}}_1(t)|}{|\mathcal{E}(t)|},P_{\mathrm{ML}}(t)=\frac{|{\mathcal{E}}_2(t)|}{|\mathcal{E}(t)|}.$$

$$P_{F}(t)\leq\Bigg{(}\frac{q^{L}-\frac{1}{q}}{q^{L}-1}\Bigg{)}\frac{q^{-(L+1)(t_{\max}-t)}}{q-1}.$$

$$|{\mathcal{B}}_S|\le q^{\frac{N}{{\log }_{2}q}-\frac{\epsilon N}{2}+\frac{1}{2}}|{\mathcal{E}}_S|.$$

$$P_e(t)=\frac{|{\mathcal{B}}_S|}{|{\mathcal{E}}_S|}\le q^{L(\frac{N}{{\log }_2{q}^L}-\frac{\epsilon N}{2}+\frac{1}{2})}.$$

$$P_F(t)=\frac{{\int }_{{\mathcal{E}}_1(t)}\varphi (\mathbf{x})d\mathbf{x}}{{\int }_{\mathcal{E}(t)}\varphi (\mathbf{x})d\mathbf{x}},P_{\mathrm{ML}}(t)=\frac{{\int }_{{\mathcal{E}}_2(t)}\varphi (\mathbf{x})d\mathbf{x}}{{\int }_{\mathcal{E}(t)}\varphi (\mathbf{x})d\mathbf{x}},$$

$${\mathbf{S}}^{(l)}(t)={\mathbf{H}}^{(l)}(t)\cdot {\mathbf{F}}^{(l)}(t)\cdot \mathbf{D}(t)\cdot \mathbf{Y}(t),$$

$${\mathbf{S}}^{(l)}(t)\cdot \mathbf{missing}{v}^{\mathsf{T}}=({\mathbf{H}}^{(l)}(t)\cdot {\mathbf{F}}^{(l)}(t)\cdot \mathbf{D}(t)\cdot \mathbf{Y}(t))\cdot {\mathbf{v}}^{\mathsf{T}}=0\forall l\in [L].$$

$$({\mathbf{H}}^{(l)}(t)\cdot {\mathbf{F}}^{(l)}(t))\cdot {\mathbf{v}}^{\mathsf{T}}=0\forall l\in [L].$$

$$\underset{\mathbf{H}}{\underbrace{\left(\begin{array}{cccc}{v}_1& v_2& \cdots & v_t\\ v_1\cdot {\alpha }_{j_1}& v_2\cdot {\alpha }_{j_2}& \cdots & v_t\cdot {\alpha }_{j_t}\\ v_1\cdot {\alpha }_{j_1}^2& v_2\cdot {\alpha }_{j_2}^2& \cdots & v_t\cdot {\alpha }_{j_t}^2\\ ⋮& ⋮& & ⋮\\ v_1\cdot {\alpha }_{j_1}^{(\rho -1)}& v_2\cdot {\alpha }_{j_2}^{(\rho -1)}& \cdots & v_t\cdot {\alpha }_{j_t}^{(\rho -1)}\end{array}\right)}}\underset{{\mathbf{f}}^{(l)}}{\underbrace{\left(\begin{array}{c}{f}_{1,l}\\ f_{2,l}\\ ⋮\\ f_{t,l}\end{array}\right)}}=0.$$

$$\mathbf{H}\cdot \mathbf{F}=0,$$

$$\mathbf{v}\cdot \mathbf{\Phi }=0,$$

$$\mathbf{\Phi }\triangleq \left(\begin{array}{cccccccccc}{f}_{1,1}& \cdots & f_{1,L}& ({\alpha }_{j_1}{f}_{1,1})& \cdots & ({\alpha }_{j_1}{f}_{1,L})& \cdots & ({\alpha }_{j_1}^{(\rho -1)}{f}_{1,1})& \cdots & ({\alpha }_{j_1}^{(\rho -1)}{f}_{1,L})\\ f_{2,1}& \cdots & f_{2,L}& ({\alpha }_{j_2}{f}_{2,1})& \cdots & ({\alpha }_{j_2}{f}_{2,L})& \cdots & ({\alpha }_{j_2}^{(\rho -1)}{f}_{2,1})& \cdots & ({\alpha }_{j_2}^{(\rho -1)}{f}_{2,L})\\ ⋮& & ⋮& ⋮& & ⋮& & ⋮& & ⋮\\ f_{t,1}& \cdots & f_{t,L}& ({\alpha }_{j_t}{f}_{t,1})& \cdots & ({\alpha }_{j_t}{f}_{t,L})& \cdots & ({\alpha }_{j_t}^{(\rho -1)}{f}_{t,1})& \cdots & ({\alpha }_{j_t}^{(\rho -1)}{f}_{t,L})\end{array}\right).$$

$${\mathbf{H}}_w\cdot {\mathbf{F}}_w=0.$$

$${\mathbf{\Phi }}_w$$

$$\tilde{\mathbf{v}}\cdot {\mathbf{\Phi }}_w=0.$$

$${\mathbf{\Phi }}_w\cdot \mathbf{u}=0.$$

$${\mathbf{\alpha }}^{(k)}\odot {\mathbf{f}}^{(l)}\triangleq ({\alpha }_1^{(k-1)}{f}_{1,l},{\alpha }_2^{(k-1)}{f}_{2,l},\dots ,{\alpha }_w^{(k-1)}{f}_{w,l}{)}^{\mathsf{T}}.$$

$$\left({\mathbf{\alpha }}^{(1)}\odot {\mathbf{f}}^{(1)},\dots ,{\mathbf{\alpha }}^{(1)}\odot {\mathbf{f}}^{(L)},{\mathbf{\alpha }}^{(2)}\odot {\mathbf{f}}^{(1)},\dots ,{\mathbf{\alpha }}^{(2)}\odot {\mathbf{f}}^{(L)},\dots ,{\mathbf{\alpha }}^{(\rho )}\odot {\mathbf{f}}^{(1)}\dots ,{\mathbf{\alpha }}^{(\rho )}\odot {\mathbf{f}}^{(L)}\right).$$

$$\mathbf{g}\triangleq {\tilde{u}}_1({\mathbf{\alpha }}^{(1)}\odot {\mathbf{f}}^{(1)})+{\tilde{u}}_2({\mathbf{\alpha }}^{(2)}\odot {\mathbf{f}}^{(1)})+\cdots +{\tilde{u}}_{\rho }({\mathbf{\alpha }}^{(\rho )}\odot {\mathbf{f}}^{(1)}).$$

Full text

I Introduction and Main Result

We consider the problem of computing $\mathbf{A}^{\mathsf{T}}\mathbf{B}$ for two matrices $\mathbf{A}\in\mathbb{F}^{s\times r}$ and $\mathbf{B}\in\mathbb{F}^{s\times r^{\prime}}$ (for an arbitrary field $\mathbb{F}$)111Some results in this paper will apply to specific fields and this will be clarified later. in a distributed fashion with $N$ worker nodes using a coded matrix multiplication scheme [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] To keep the presentation clear, we will focus on one class of codes, namely Polynomial codes, and explain our results in relation to the Polynomial codes [1]; notwithstanding, our results also apply to Entangled Polynomial codes [2] and PolyDot codes [3]. We assume that the matrices $\mathbf{A}$ and $\mathbf{B}$ are split into $m$ subblocks and $n$ subblocks, respectively. These subblocks are encoded using a Polynomial code [2]. Each worker node performs a matrix multiplication and returns a matrix with a total of $L=\frac{rr^{\prime}}{mn}$ elements (from $\mathbb{F}$) to the master node.

Our main interest is in the fault-tolerant setup where some of the $N$ worker nodes return erroneous values. We say that an error pattern of Hamming weight $t$ has occurred if $t$ worker nodes return matrices that contain some erroneous values. The main idea in the Polynomial codes, Entangled Polynomial codes and PolyDot codes is to encode the subblocks of $\mathbf{A}$ and $\mathbf{B}$ in a clever way such that the matrix product returned by the worker nodes are symbols of a codeword of a Reed-Solomon (RS) code over $\mathbb{F}$. The properties of an RS code are then used to obtain bounds on the error-correction capability of the scheme.

The main contribution of this work relies on the observation that Polynomial codes, Entangled Polynomial codes, and PolyDot codes are not just RS codes, but an Interleaved Reed-Solomon (IRS) code which consists of several RS codes that can be collaboratively decoded (see Section III or [12] for a formal definition). This additional structure provides the opportunity for collaborative decoding of multiple RS codes involved in such coded matrix multiplication schemes. Such a collaborative decoding, for which efficient multi-sequence shift-register (MSSR) based decoding algorithms exist [13], provides a practical decoder with quadratic complexity in $t$, while potentially nearly doubling the decoding radius.

The main results of this paper and their relation to the existing results are as follows. In [2], it is shown that any error pattern with Hamming weight $t$ can be corrected if $t\leq\lfloor\frac{N-K}{2}\rfloor$ where $K=mn$ is the effective dimension of the Polynomial code. Very recently, Dutta et al. in [3] showed that when $\mathbb{F}=\mathbb{R}$ (the real field) and error values are randomly distributed according to a Gaussian distribution, with probability $1$ all error patterns of Hamming weight $t\leq N-K-1$ can be corrected. To attain this bound, [3] uses a decoding algorithm which is similar in spirit to exhaustive maximum likelihood decoding with a complexity that is O\big{(}LN^{\min\{t,N-t\}}\big{)}. This can be prohibitive for many practical values of $N$ and $t$. In [3], it is suggested that in practice, the performance of ML decoding can be approximated by algorithms with polynomial complexity in $N$ such as the $\ell_{1}$-minimization algorithm [14]. However, there is no proof (nor evidence) that such algorithms can correct all error patterns of Hamming weight up to $N-K-1$ with probability $1$. Indeed, as we will show in this work, the standard $\ell_{1}$-minimization based decoding algorithm [14] fails to correct all error patterns of Hamming weight up to $N-K-1$ with a non-zero probability.

In this work, we show that we can use the MSSR decoding algorithm of [13] for decoding Polynomial codes with the complexity of O\big{(}Lt^{2}+N\big{)}. For this algorithm, we will show that when $\mathbb{F}=\mathbb{F}_{q}$ (a finite field with $q$ elements), for $\lfloor\frac{N-K}{2}\rfloor<t\leq\frac{L}{L+1}(N-K)$, all but a fraction $\gamma(t)$ of the error patterns of Hamming weight $t$ can be corrected where $\gamma(t)\rightarrow 0$ as $q\rightarrow\infty$. In particular, the convergence of $\gamma(t)$ to zero is exponentially fast in $L$, i.e., $\gamma(t)=q^{-\Omega(L)},$ for $\lfloor\frac{N-K}{2}\rfloor<t\leq\frac{L}{L+1}(N-K)$. In addition, when $\mathbb{F}=\mathbb{R}$, by extending the results of [13] and [15] to the real field and using the results of [3], we will show that for $L\geq N-K-1$ and ${\lfloor\frac{N-K}{2}\rfloor<t\leq N-K-1}$, all error patterns of Hamming weight $t$ can be corrected with probability $1$, under the random Gaussian error model previously considered in [3].

In a nutshell, our results show that with a probability arbitrarily close to $1$ (or respectively, with probability $1$), all error patterns of Hamming weight up to $\frac{L}{L+1}(N-K)$, which can be made arbitrarily close to $N-K-1$ for sufficiently large $L$, can be corrected for sufficiently large finite fields (or respectively, the real field). Not only does this indicate a substantial increase in the error-correction radius with provable guarantees when compared to the results in [2], but it also shows that the Dutta et al.’s upper bound in [3] can be achieved with a practical decoder with a quadratic complexity in the number of faulty worker nodes ($t$). This improvement in complexity is the result of collaboratively decoding the IRS code instead of separately decoding the RS codes using a maximum likelihood decoder as is done in [3].

II Review of Polynomial Codes for Distributed Matrix Multiplication

II-A Notation

Throughout the paper, we denote matrices by boldface capital letters, e.g., $\mathbf{A}$, and denote vectors by boldface small letters, e.g., $\mathbf{a}$. For an integer $i\geq 1$, we denote $\{1,\dots,i\}$ by $[i]$, and for two integers $i$ and $j$ such that $i<j$, we denote $\{i,i+1,\dots,j\}$ by $[i,j]$. We use the short notation $((f(i,j))_{i\in[m],j\in[n]})$ to represent an $m\times n$ matrix whose entry $(i,j)$ is $f(i,j)$, where $f(i,j)$ is a function of $i$ and $j$. We occasionally use the compact notation $(\mathbf{a}_{1},\mathbf{a}_{2},\dots,\mathbf{a}_{n})$ to represent an $m\times n$ matrix whose columns are the column-vectors $\mathbf{a}_{1},\mathbf{a}_{2},\dots,\mathbf{a}_{n}$, each of length $m$. Similarly, sometimes we use the compact notation $(\mathbf{a}_{1};\mathbf{a}_{2};\dots;\mathbf{a}_{m})$ to represent an $m\times n$ matrix whose rows are the row-vectors $\mathbf{a}_{1},\mathbf{a}_{2},\dots,\mathbf{a}_{m}$, each of length $n$. We also denote by $\mathbf{A}(i,:)$ and $\mathbf{A}(:,j)$ the $i$th row and the $j$th column of a matrix $\mathbf{A}$, respectively. A vector or a matrix with a $\wedge$ above is an estimate.

II-B Polynomial Codes

In this section, we review the Polynomial codes of Yu, Maddah-Ali and Avestimehr [1] for distributed matrix multiplication. Consider the problem of computing $\mathbf{A}^{\mathsf{T}}\mathbf{B}$ in a distributed fashion for two matrices $\mathbf{A}\in\mathbb{F}^{s\times r}$ and $\mathbf{B}\in\mathbb{F}^{s\times r^{\prime}}$ for an arbitrary field $\mathbb{F}$. In the scheme of Polynomial codes in [1], the master node distributes the task of matrix multiplication among $N$ worker nodes as follows.

The columns of $\mathbf{A}$ and $\mathbf{B}$ are first partitioned into $m$ partitions $\mathbf{A}_{0},\mathbf{A}_{1},\dots,\mathbf{A}_{m-1}$ of equal size $\frac{r}{m}$ and $n$ partitions $\mathbf{B}_{0},\mathbf{B}_{1},\dots,\mathbf{B}_{n-1}$ of equal size $\frac{r^{\prime}}{n}$, respectively,

[TABLE]

Let $x_{1},x_{2},\ldots,x_{N}$ be $N$ distinct elements in $\mathbb{F}$. For two parameters $\alpha,\beta\in[N]$, let $\tilde{\mathbf{A}}_{i}$ and $\tilde{\mathbf{B}}_{i}$ be matrices defined by,

[TABLE]

The dimensions of the matrices $\tilde{\mathbf{A}}_{i}$ and $\tilde{\mathbf{B}}_{i}$ are $s\times\frac{r}{m}$ and $s\times\frac{r^{\prime}}{n}$, respectively.

The $i$th worker node computes the smaller matrix product $\tilde{\mathbf{C}}_{i}$ given the values of $\tilde{\mathbf{A}}_{i}$ and $\tilde{\mathbf{B}}_{i}$,

[TABLE]

The parameters $\alpha$ and $\beta$ are chosen carefully such that for each pair $(j,k)$ the corresponding exponent of $x_{i}$ (i.e., $j\alpha+k\beta$) is distinct. For instance, one such choice for $\alpha$ and $\beta$ is $\alpha=1$ and $\beta=m$. In this case, the $i$th worker node essentially evaluates $\mathbf{P}(x)$ at $x=x_{i}$ and returns $\mathbf{P}(x_{i})$, where

[TABLE]

The coefficients in the polynomial $\mathbf{P}(x)$ are the $mn$ uncoded symbols of the product $\tilde{\mathbf{C}}_{i}$ in (1). The crux of the Polynomial code is that the vector of coded symbols $\left(\mathbf{P}(x_{1}),\ldots,\mathbf{P}(x_{N})\right)=(\tilde{\mathbf{C}}_{1},\tilde{\mathbf{C}}_{2},\cdots,\tilde{\mathbf{C}}_{N})$ can be considered as a codeword of a Reed-Solomon (RS) code. If $N$ worker nodes are available in the distributed system, a Polynomial code essentially evaluates the polynomial $\mathbf{P}(x)$ at $N$ points of the field $\mathbb{F}$; any $mn$ of which can recover the coefficients which can be put together to recover the matrix product. The minimum number of worker nodes that need to compute and return the correct evaluations of $\mathbf{P}(x)$ for the master node to be able to successfully recover the matrix product $\mathbf{A}^{\mathsf{T}}\mathbf{B}$ is called the recovery threshold. Viewing the recovery process of a Polynomial code as a polynomial interpolation operation, it can be seen that the recovery threshold of the Polynomial code is $mn$ [1].

III Polynomial Codes are Interleaved Reed-Solomon Codes

Definition 1**.**

Generalized Reed-Solomon (GRS) Codes: Let $\mathbf{m}=(m_{0},m_{1},\ldots,m_{K-1})$ and let the associated polynomial $m(x)$ be defined as $m(x):=m_{0}+m_{1}x+\ldots+m_{K-1}x^{K-1}$. Further, let $\mathbf{c}=(c_{0},c_{1},\dots,c_{N-1})$, $\boldsymbol{\alpha}=(\alpha_{0},\alpha_{1},\ldots,\alpha_{N-1})$ and $\mathbf{v}=(v_{0},v_{1},\ldots,v_{N-1})$ be three row vectors such that $c_{i},\alpha_{i},v_{i}\in\mathbb{F}$, $v_{i}\neq 0$, and $\alpha_{i}\neq\alpha_{j}$. A Generalized Reed-Solomon (GRS) code $\mathcal{C}$ over $\mathbb{F}$ of length $N$, dimension $K$, evaluation points $\boldsymbol{\alpha}$, weight vectors $\mathbf{v}$, denoted by GRS$(\mathbb{F},N,K,\boldsymbol{\alpha},\mathbf{v})$, is the set of all row-vectors (codewords) $\mathbf{c}=(v_{0}m(\alpha_{0}),v_{1}m(\alpha_{1}),\ldots,v_{N-1}m(\alpha_{N-1}))$, i.e., $c_{i}=v_{i}m(\alpha_{i})$. Equivalently, a GRS code is also the set of codewords $\mathbf{c}$ such that for all $i\in[0,N-K-1]$, $\sum_{j=0}^{N-1}u_{j}c_{j}(\alpha_{i})^{j}=0$, where $u_{i}^{-1}=v_{i}\prod\limits_{j\neq i}(\alpha_{i}-\alpha_{j})$. The minimum distance of such a GRS code is $d_{\min}=N-K+1$.

Reed-Solomon (RS) codes are a special case of GRS codes with $v_{i}=1,u_{i}=1,\forall i\in[0,N-1]$. For finite fields and the complex field, an $\boldsymbol{\alpha}$ exists such that $v_{i}=1$ and $u_{i}=1$, $i\in[0,N-1]$. However for the real field, $u_{i}$ and $v_{i}$ cannot be simultaneously set to 1 and, hence, it is required to consider GRS codes.

Definition 2**.**

Interleaved Generalized Reed-Solomon (IGRS) Codes [12]: Let $\{\mathcal{C}^{(l)}\}_{l\in[L]}$ be a collection of $L$ GRS codes $\mathcal{C}^{(l)}\triangleq\emph{\text{RS}}(\mathbb{F},N,K^{(l)},\boldsymbol{\alpha},\boldsymbol{u})$, each of length $N$ over a field $\mathbb{F}$, where the dimension and minimum distance of the $l$th GRS code are $K^{(l)}$ and $d^{(l)}$, respectively. Then, an Interleaved Generalized Reed-Solomon (IGRS) code $\mathcal{C}_{\rm{IGRS}}$ is the set of all $L\times N$ matrices $(\mathbf{c}^{(1)};\mathbf{c}^{(2)};\dots;\mathbf{c}^{(L)})$ where $\mathbf{c}^{(l)}\in\mathcal{C}^{(l)}$ for $l\in[L]$ [13]. If all the $L$ GRS codes $\mathcal{C}^{(l)}$ are equivalent, i.e., $\mathcal{C}^{(l)}=\mathcal{C}$ for all $l\in[L]$, the IGRS code $\mathcal{C}_{\mathrm{IRS}}$ is called homogeneous.

The chief observation in this work is that the Polynomial codes, Entangled Polynomial codes, and PolyDot codes are IGRS codes. Here, we formally prove this observation for the Polynomial codes. We shall henceforth refer to GRS codes and IGRS codes as RS codes and IRS codes, respectively.

Theorem 3**.**

A Polynomial code is an IRS code.

Proof.

Let $\mathbf{W}$ be an $a\times b$ matrix with entries from $\mathbb{F}$, and let $\Gamma:\mathbb{F}^{a\times b}\rightarrow\mathbb{F}^{ab}$ denote a vectorizing operator which reshapes a matrix $\mathbf{W}$ into a column-vector $\mathbf{w}=(w_{1},\dots,w_{ab})^{\mathsf{T}}$, i.e., $\Gamma(\mathbf{W})=\mathbf{w}$, such that $w_{(i-1)b+j}=\mathbf{W}(i,j)$, where $\mathbf{W}(i,j)$ is the element $(i,j)$ of $\mathbf{W}$.

Let $\tilde{\mathbf{C}}_{i}(p,q)$ be the element $(p,q)$ of the matrix $\tilde{\mathbf{C}}_{i}$,

[TABLE]

Consider the $\frac{rr^{\prime}}{mn}\times N$ matrix $\mathbf{D}\triangleq(\Gamma(\tilde{\mathbf{C}}_{1}),\Gamma(\tilde{\mathbf{C}}_{2}),\dots,\Gamma(\tilde{\mathbf{C}}_{N}))$, where the $i$th column of $\mathbf{D}$, namely $\Gamma(\tilde{\mathbf{C}}_{i})$, is obtained by applying the vectorizing operator $\Gamma$ to $\tilde{\mathbf{C}}_{i}$. Let $(p_{i},q_{i})$ be the unique pair $(p,q)$ such that $i=(p-1)\frac{r^{\prime}}{n}+q$. Then, the element $(i,j)$ of $\mathbf{D}$ is $\tilde{\mathbf{C}}_{j}{(p_{i},q_{i})}$, and accordingly, the $i$th row of $\mathbf{D}$ is given by $[\tilde{\mathbf{C}}_{1}{(p_{i},q_{i})},\tilde{\mathbf{C}}_{2}{(p_{i},q_{i})},\ldots,\tilde{\mathbf{C}}_{N}{(p_{i},q_{i})}]$, which is a codeword of an RS code. Thus the matrix $\mathbf{D}$ is a codeword of an IRS code with $L=\frac{rr^{\prime}}{mn}$. In particular, the $i$th worker node computes $\tilde{\mathbf{C}}_{i}$ that has dimension $\frac{r}{m}\times\frac{r^{\prime}}{n}$. It is evident from (3) that the element $(p,q)$ of $\tilde{\mathbf{C}}_{i}$ is the message polynomial $\sum\limits_{j=0}^{m-1}\sum\limits_{k=0}^{n-1}[\mathbf{A}_{j}^{\mathsf{T}}\mathbf{B}_{k}]{(p,q)}x^{j+km}$ evaluated at $x_{i}$. Thus, $\tilde{\mathbf{C}}_{i}$ contains $\frac{rr^{\prime}}{mn}$ RS codes evaluated at $x_{i}$ by the $i$th worker node. That is, the computations returned by the $i$th worker node constitute the $i$th column of an IRS code with $N$ being the number of worker nodes and $L=\frac{rr^{\prime}}{mn}$ being the number of RS codes. This shows that a Polynomial code is a homogeneous IRS code with $K^{(l)}=mn$ for $l\in[L]$. ∎

III-A Error Matrix and Error Models

We consider the case when the worker nodes introduce additive errors in their computation. Let ${\mathbf{E}_{i}\in\mathbb{F}^{\frac{r}{m}\times\frac{r^{\prime}}{n}}}$ denote the error matrix introduced by the $i$th worker node. Then the master node receives the set of matrices $\tilde{\mathbf{R}}_{i}$, for $i\in[N]$ where $\tilde{\mathbf{R}}_{i}=\tilde{\mathbf{C}}_{i}\oplus\tilde{\mathbf{E}}_{i}$. Let $\mathbf{R}$ be the $\frac{rr^{\prime}}{mn}\times N$ matrix of values received by the master node where the $i$th column of $\mathbf{R}$ is given by $\Gamma(\tilde{\mathbf{R}}_{i})$, and let $\mathbf{E}$, referred to as the error matrix, be the $\frac{rr^{\prime}}{mn}\times N$ matrix of error values where the $i$th column of $\mathbf{E}$ is given by $\Gamma(\tilde{\mathbf{E}}_{i})$. Then, $\mathbf{R}=\mathbf{D}\oplus\mathbf{E}$ where $\mathbf{D}$ is a codeword of an IRS code. If the $i$th worker node returns erroneous values, then the $i$th column of $\mathbf{R}$ will contain errors. Thus, the original problem of fault-tolerant distributed matrix multiplication reduces to the problem of decoding $\mathbf{D}$ from $\mathbf{R}$.

Definition 4**.**

The Hamming weight of the matrix $\mathbf{E}$ denoted by $W_{H}(\mathbf{E})$ is defined as the number of non-zero columns in $\mathbf{E}$.

We consider two different error models. First, we consider the Uniform Random Error for Finite Fields (UREF) model where the non-zero columns of the error matrix $\mathbf{E}$ are assumed to be uniformly distributed over all the non-zero vectors in $\mathbb{F}^{L}_{q}$ for a finite field $\mathbb{F}_{q}$. We further extend this model to the real field $\mathbb{R}$ where each non-zero entry in the error matrix $\mathbf{E}$ is assumed to be an independently and identically distributed Gaussian random variable (with arbitrary mean and variance). This model is referred to as the Gaussian Random Error (GRE) model.

III-B Decoding and Error Events

Let $\psi:\mathbb{F}^{L\times N}\rightarrow\{\mathcal{C}_{\rm{IRS}},F\}$ be the decoding function, where $F$ is a symbol that denotes decoding failure. A decoding error is said to have occurred if $\psi(\mathbf{R})\neq\mathbf{D}$. An undetected decoding error is said to have occurred if $\psi(\mathbf{R})\neq\mathbf{D}$ and $\psi(\mathbf{R})\neq F$, whereas a decoding failure is said to have occurred if $\psi(\mathbf{R})=F$.

IV Collaborative Decoding of Interleaved Reed-Solomon Codes

Simultaneous decoding of all the RS codes in an IRS code is known as collaborative decoding. As shown in [12] and [13], collaborative decoding of IRS codes has certain advantages. In particular, when burst errors occur, they occur on the same column of the IRS code. Hence, multiple RS codewords share the same error positions. Note that an IRS code is actually a set of RS codes stacked together, each of which yields a set of syndrome equations. Intuitively, when burst errors occur, the error locator polynomials are more or less the same for all the RS codes but the number of syndrome equations increases with the number of stacked RS codes. This implies that a much larger set of errors can be corrected. This is because the rank of the stacked syndrome matrix is greater than or equal to the rank of the individual syndrome matrices, thus giving rise to the possibility of a greater decoding radius than the unique decoding bound of $\frac{1-R}{2}$, where $R$ is the code rate. More specifically, it was shown by Schmidt et al. in [12] that when a set of $L$ RS codes are collaboratively decoded, except for a small probability of failure and a small probability of error (discussed in Section VI), the fraction of errors that can be corrected can be as large as $\frac{L}{L+1}(1-R)$.

V Decoding Algorithms

V-A Collaborative Peterson’s Algorithm

In this section, we propose a collaborative version of the Peterson’s algorithm [16] to correct up to $t\leq t_{\text{max}}\triangleq\frac{L}{L+1}(N-K)$ errors.

Consider $t$ non-zero errors in columns $j_{1},j_{2},\dots,j_{t}$ of the matrix $\mathbf{R}$ (i.e., the indices of the non-zero columns of the error matrix $\mathbf{E}$ are $j_{1},j_{2},\dots,j_{t}$). Let $r^{(l)}(z)\triangleq\sum_{j=0}^{N-1}u_{j}\mathbf{R}(l,j)z^{j-1}$ be the modified (multiplying component-wise by $u_{j}$) received polynomial for the $l$th RS code, where $\mathbf{R}(l,j)$ is the element $(l,j)$ of the matrix $\mathbf{R}$. Then, the syndrome sequence for the $l$th RS code is given by $S^{(l)}\triangleq\{S^{(l)}_{i}\}_{i=0}^{N-K-1}$, where $S_{i}^{(l)}\triangleq\sum_{j=0}^{N-1}u_{j}\mathbf{R}(l,j)\alpha_{j}^{i}$ for $i\in[0,N-K-1]$. Define the error locator polynomial $\Lambda(z)$ as

[TABLE]

and let $\boldsymbol{\lambda}(t)=(\lambda_{t},\lambda_{t-1},\dots,\lambda_{1})^{\mathsf{T}}$ be the error locator vector associated with the error locator polynomial $\Lambda(z)$. When $t$ errors occur $\Lambda(z)$ has a degree of $t$. The syndrome matrix $\mathbf{S}^{(l)}(t)$ and a vector $\mathbf{a}^{(l)}(t)$ for the $l$th RS code are given by

[TABLE]

Now we can write the following consistent linear system of equations for the IRS code,

[TABLE]

where $\mathbf{S}_{L}(t)$, the syndrome matrix for the IRS code, is the stacked matrix of $\mathbf{S}^{(l)}(t)$ for $l\in[L]$, and $\mathbf{a}_{L}(t)$, a vector for the IRS code, is the stacked vector of $\mathbf{a}^{(l)}(t)$ for $l\in[L]$. If $t$ columns of the matrix $\mathbf{R}$ are in error, then the error locator vector $\boldsymbol{\lambda}(t)$ can be obtained by the collaborative Peterson’s algorithm, described in Algorithm 1. The complexity of computing the rank of $\mathrm{rank}(\mathbf{S}_{L}(\tau))$ is $O(L\tau^{3})$; computing $\hat{\boldsymbol{\lambda}}$ requires $O(\tau^{3})$ operations if the structure of $\mathbf{S}_{L}(\tau)$ is not exploited, and the Chien search has a complexity of $O(N)$. Since we have to consider all values of $\tau\in[t_{\max}]$, the overall complexity is $O(Lt_{\max}^{4}+N)$.

Definition 5**.**

($t$-valid polynomial $\Lambda(z)$): A polynomial $\Lambda(z)$ over $\mathbb{F}$ is called $t$-valid if it is a polynomial of degree $t$ and possesses exactly $t$ distinct roots in $\mathbb{F}$.

V-B Multiple Sequence Shift Register algorithm

A more computationally efficient decoding algorithm to achieve error correction up to $t\leq t_{\mathrm{max}}=\frac{L}{L+1}(N-K)$ is the Multiple Sequence Shift Register (MSSR) algorithm proposed by Schmidt et al. in [15]. This algorithm has a complexity of $O(Lt^{2}+N)$. The MSSR algorithm, reviewed here for completeness, is described in Algorithm 2.

It can be seen that in the absence of numerical round-off errors, the outputs of the collaborative Peterson’s algorithm and the MSSR algorithm are identical for every $\mathbf{R}$ since both of them compute the solution to (5).

VI Analysis of probability of failure and error for finite fields ($\mathbb{F}=\mathbb{F}_{q}$)

In Section III, we showed that Polynomial codes are IRS codes. Hence the fault tolerance of the Polynomial codes can be analyzed using similar techniques for IRS codes. In this section, we consider the uniformly random error model for finite fields (UREF), defined in Section III-A, which was originally considered in [12]. In particular, we define the error events

[TABLE]

Since the outputs of the collaborative Peterson’s algorithm and the MSSR algorithm are identical for every $\mathbf{R}$, both algorithms have the same probability of failure and the same probability of undetected error. We denote by $P_{F}(t)$ and $P_{\rm ML}(t)$ the probability of failure and the probability of undetected error, respectively, given that $W_{H}(\mathbf{E})=t$. Under the UREF model, $P_{F}(t)$ and $P_{\rm ML}(t)$ are given by [12]

[TABLE]

VI-A Probability of Failure

A necessary condition for the failure of both the collaborative Peterson’s algorithm and the MSSR algorithm is that the matrix $\mathbf{S}_{L}(t)$ is not full rank, as shown in [12]. To calculate an upper bound on $P_{F}(t)$, we refer to the analysis by schmidt et al. in [12], and recall the following result from [12].

Theorem 6**.**

[12*, Theorem 7]**

Under the UREF model, for all $t\leq t_{\max}=\frac{L}{L+1}(N-K)$,*

[TABLE]

By the result of Theorem 6, it can be readily seen that for all $t<t_{\rm max}$, $P_{F}(t)$ diminishes as q^{-\Omega\big{(}L\big{)}} and for $t=t_{\rm max}$, $P_{F}(t)$ decays as $q^{-1}$ .

VI-B Probability of Undetected Error

As shown in [12, Theorem 5], the MSSR algorithm has the Maximum Likelihood (ML) certificate property, i.e., whenever the decoder of [15] does not fail, it yields the ML solution, namely the codeword at minimum Hamming distance from the received word. The collaborative Peterson’s algorithm has the same ML certificate property as well. An error matrix $\mathbf{E}$ with $W_{H}(\mathbf{E})=t$ is said to be a bad error matrix of Hamming weight $t$ if there exists a non-zero codeword $\mathbf{D}\in\mathcal{C}_{\rm IRS}$ such that $W_{H}(\mathbf{D}\ominus\mathbf{E})\leq t$.

We now use a result from [17, Page 141] without proof.

Lemma 7**.**

[17*, Page 141]**

Let $\mathcal{C}\subseteq\{0,1,\cdots q-1\}^{N}$ be a code with relative distance $\delta=d_{\min}/N$, and let $S\subseteq[N]$ be such that $|S|=(1-\gamma)N$, where $0<\gamma\leq\delta-\varepsilon$ for some $\varepsilon>0$. Let $\mathcal{E}_{S}$ be the set of all error vectors with support $S^{c}$, and let $\mathcal{B}_{S}$ be the set of all bad error vectors with support $S^{c}$. Then,*

[TABLE]

Theorem 8**.**

Under the UREF model, for all $t\leq N-K-1$ (and in particular, for all $t\leq t_{\max}=\frac{L}{L+1}(N-K)$), $P_{\rm ML}(t)\to 0$ as $q^{L}\to\infty$.

Proof.

It is easy to see that an IRS code can be viewed as a single code over $\mathbb{F}_{q^{L}}$, i.e. $\mathcal{C}_{\rm IRS}$ is a \big{(}\mathbb{F}_{q^{L}},N,K,N-K+1\big{)} code. Lemma 7 holds for a single code and, hence, can be applied to $\mathcal{C}_{\rm IRS}$ with $q$ being replaced by $q^{L}$. Since the upper bound in Lemma 7 depends only on the cardinality of $\mathcal{E}_{S}$, it follows that the probability of having a bad error matrix with $W_{H}(\mathbf{E})=t$ for the \big{(}\mathbb{F}_{q^{L}},N,K,N-K+1\big{)} code (replacing $q$ by $q^{L}$ since $\mathcal{C}_{\rm IRS}$ is over $q^{L}$) which we denote by $P_{e}(t)$ is upper bounded by

[TABLE]

By setting $\delta=\frac{N-K+1}{N}$ and $\varepsilon=\frac{2}{N}$, it is easy to see that $P_{e}(t)\rightarrow 0$ as $q^{L}\to\infty$. For this choice of $\delta$ and $\varepsilon$, it follows that $\gamma\leq\delta-\varepsilon=\frac{N-K-1}{N}$, which implies that (8) holds for all $t\leq N-K-1$.

Note that the algorithms in Section V have the ML certificate property. Note, also, that the fraction of error matrices that give rise to an undetected error is upper bounded by the fraction of bad error matrices. This is simply because without a bad error matrix of Hamming weight up to $(\delta-\varepsilon)N$, an undetected error cannot occur. Thus, $P_{\rm ML}(t)\leq P_{e}(t)$. Since $P_{e}(t)$ vanishes as $q^{L}\to\infty$, then $P_{\rm ML}(t)$ vanishes as $q^{L}\to\infty$. Moreover, $N$ and $K$ are fixed and finite, and hence, $\sum_{t=1}^{N-K-1}P_{\rm ML}(t)\to 0$ as $q^{L}\to\infty$. ∎

VII Analysis of probablity of failure and probability of error for the real field

In this section, we analyze the probability of failure and probability of error under the GRE model when the computations are performed over the real field. In particular, we consider the case that the error values are independently and identically distributed standard Gaussian random variables (with zero mean and unit variance). Note, however, that this assumption does not limit the generality of the results, and is made for the ease of exposition only. For this model, conditioned on $t$ errors occurring, the probability of failure ($P_{F}(t)$) and the probability of undetected error ($P_{\mathrm{ML}}(t)$) are given by

[TABLE]

where $\mathcal{E}_{1}(t),\mathcal{E}_{2}(t),\mathcal{E}(t)$ are defined as in (6), and $\phi(\mathbf{x})$ is the probability density function of an $Lt$-dimensional standard Gaussian random vector (with zero-mean vector and identity covariance matrix).

VII-A Probability of Failure

It should be noted that the results of [12] for finite fields cannot be directly extended to the real field, simply because the counting arguments used in [12] for finite fields do not carry over to the real field. In this section, we propose a new approach to derive the probability of failure for the real field case.

For simplifying the notation, hereafter, we use $\rho\triangleq N-K-t$. Suppose that $t\leq t_{\rm max}=\frac{L}{L+1}(N-K)$ errors occur at positions $j_{1},j_{2},\cdots,j_{t}$ with values $e_{j_{1}}^{(l)},e_{j_{2}}^{(l)},\cdots,e_{j_{t}}^{(l)}$ for the $l$th RS code. Recall the syndrome matrix $\mathbf{S}^{(l)}(t)$ for the $l$th RS code (see (4)). As shown in [12], $\mathbf{S}^{(l)}(t)$ can be decomposed as

[TABLE]

where $\mathbf{H}^{(l)}(t)\triangleq(\alpha^{(i-1)}_{j_{k}})_{i\in[\rho],k\in[t]}$ is an $\rho\times t$ matrix, $\mathbf{F}^{(l)}(t)\triangleq\text{diag}((e_{j_{i}}^{(l)})_{i\in[t]})$ is a $t\times t$ diagonal matrix, $\mathbf{D}(t)\triangleq\text{diag}((\alpha_{j_{i}})_{i\in[t]})$ is a $t\times t$ diagonal matrix, and $\mathbf{Y}(t)\triangleq(\alpha^{(k-1)}_{j_{i}})_{i\in[t],k\in[t]}$ is a $t\times t$ matrix.

Theorem 9**.**

Under the GRE model, for all $t\leq t_{\max}=\frac{L}{L+1}(N-K)$, $P_{F}(t)=0$. In particular, for $L\geq N-K-1$, for all $t\leq N-K-1$, $P_{F}(t)=0$.

Proof.

The decoding algorithms described in Section V fail when the stacked matrix $\mathbf{S}_{L}(t)$ defined in (5) is rank deficient, i.e., there exists a non-zero row-vector $\mathbf{v}$ such that $\mathbf{S}_{L}(t)\cdot\mathbf{v}^{\mathsf{T}}=0$. Alternatively, $\mathbf{S}_{L}(t)$ is rank deficient iff there exists a non-zero row-vector $\mathbf{v}$ such that

[TABLE]

Since $\mathbf{D}(t)$ and $\mathbf{Y}(t)$ are invertible, the condition (9) holds iff there is a non-zero row-vector $\mathbf{v}$ such that

[TABLE]

Let $\mathbf{v}=(v_{1},v_{2},\dots,v_{t})$, and let $f_{i,l}\triangleq e^{(l)}_{j_{i}}$ for all $i\in[t]$. Expanding (10), it is easy to see that

[TABLE]

Combining the condition (11) for all the RS codes in the IRS code (for all $l\in[L]$), it holds that

[TABLE]

where $\mathbf{H}$ is defined in (11), and $\mathbf{F}\triangleq(\mathbf{f}^{(1)},\mathbf{f}^{(2)},\dots,\mathbf{f}^{(L)})$ is a $t\times L$ matrix where $\mathbf{f}^{(l)}$ for $l\in[L]$ is defined in (11). Alternatively, (12) can be written as

[TABLE]

where $\mathbf{\Phi}$ is a $t\times\rho L$ matrix given by

[TABLE]

Let $\mathcal{F}$ be the set of all $t\times L$ matrices $\mathbf{F}=(f_{i,l})_{i\in[t],l\in[L]}$ for each of which the condition (12) holds for some non-zero vector $\mathbf{v}$. We need to show that $\mathcal{F}$ is a set of measure zero.

We consider two cases as follows: (i) $t\leq L$, and (ii) $t>L$.

Case (i): For the condition (12) to hold, there must exist a non-zero vector $\mathbf{v}$ in the left null space of $\mathbf{F}$. It is easy to see that, under the GRE model, the set of all matrices $\mathbf{F}$ that have a row-rank of $t$ is a set of measure $1$. This implies that the set of all matrices $\mathbf{F}$ for each of which there exists some non-zero vector $\mathbf{v}$ in the left null space of $\mathbf{F}$ is a set of measure zero. Thus, for $t\leq L$, $\mathcal{F}$ is a set of measure zero.

Case (ii): For a vector $\mathbf{v}$, let the weight of $\mathbf{v}$, denoted by $\mathrm{wt}(\mathbf{v})$, be the number of non-zero elements in $\mathbf{v}$. For any integer $1\leq w\leq t$, let $\mathcal{F}_{w}$ be the set of all matrices $\mathbf{F}$ for each of which there exists a non-zero vector $\mathbf{v}$ such that $\mathrm{wt}(\mathbf{v})=w$ and the condition (12) holds.

We consider two cases as follows: (1) $w\leq\rho$, and (2) $w>\rho$. (Recall that $\rho=N-K-t$.)

1. (1)

$w\leq\rho$: Assume, without loss of generality, that $v_{1},v_{2},\cdots,v_{w}$ are the non-zero elements of $\mathbf{v}$. Let $\mathbf{H}_{w}\triangleq((v_{k}\cdot\alpha^{(i-1)}_{j_{k}})_{i\in[w],k\in[w]})$ be the $w\times w$ sub-matrix of $\mathbf{H}$ (defined in (12)) corresponding to the first $w$ rows and the first $w$ columns, and let $\mathbf{F}_{w}\triangleq((f_{i,l})_{i\in[w],l\in[L]})$ be the $w\times L$ sub-matrix of $\mathbf{F}$ corresponding to the first $w$ rows. Then, the condition (12) reduces to

[TABLE]

It is easy to see that the matrix $\mathbf{H}_{w}$ generates a Generalized Reed-Solomon code with distinct parameters $\{\alpha_{j_{i}}\}_{i\in[w]}$ and non-zero multipliers $\{v_{i}\}_{i\in[w]}$. Thus, $\mathbf{H}_{w}$ is full rank (and hence, invertible). This implies that for each $l\in[L]$ the column-vector $\mathbf{f}^{(l)}$ (defined in (11)) is an all-zero vector. Thus, every matrix in $\mathcal{F}_{w}$ for $w\leq\rho$ contains a $w\times L$ all-zero sub-matrix. In particular, every matrix in $\mathcal{F}_{w}$ for $w\leq\rho$ has at least one fixed (zero, in this case) entry. Under the GRE model, it is then easy to see that $\mathcal{F}_{w}$ for $w\leq\rho$ is a set of measure zero. 2. (2)

$w>\rho$: Assume, without loss of generality, that $v_{1},\dots,v_{w}$ are the non-zero elements of $\mathbf{v}$, and let $\tilde{\mathbf{v}}\triangleq(v_{1},v_{2},\cdots,v_{w})$. Let $\mathbf{\Phi}_{w}$ be the $w\times\rho L$ sub-matrix of $\mathbf{\Phi}$ (defined in (14)) corresponding to the first $w$ rows,

[TABLE]

Then, the condition (13) reduces to

[TABLE]

Since in (5) the number of variables must be less than the number of equations, then $w\leq t\leq\rho L$. Note that $\mathbf{\Phi}_{w}$ is a $w\times\rho L$ matrix. Thus, $\mathrm{rank}(\mathbf{\Phi}_{w})\leq w$. Moreover, there exists a non-zero vector $\tilde{\mathbf{v}}$ in the left null space of $\mathbf{\Phi}_{w}$. This implies that $\mathrm{rank}(\mathbf{\Phi}_{w})\leq w-1$. Since the row-rank and the column-rank are equal, there exists a non-zero column-vector $\mathbf{u}$ such that

[TABLE]

Let $\alpha_{i}\triangleq\alpha_{j_{i}}$ for $i\in[w]$, and let $\boldsymbol{\alpha}^{(k)}=(\alpha_{1}^{k-1},\alpha_{2}^{k-1},\cdots,\alpha_{w}^{k-1})^{\mathsf{T}}$ for $k\in[\rho]$. We define the product operator $\odot$ between the two vectors $\boldsymbol{\alpha}^{(k)}$ and $\mathbf{f}^{(l)}$ as

[TABLE]

Then, we can rewrite $\mathbf{\Phi}_{w}$ as

[TABLE]

Since $\mathbf{u}=(u_{1},\dots,u_{L},u_{L+1},\dots,u_{L+L},\dots,u_{(\rho-1)L+1},\dots,u_{(\rho-1)L+L})\neq 0$, there exist $l\in[L]$ and $k\in[\rho]$ such that $u_{(k-1)L+l}$ is non-zero. Assume, without loss of generality, that $u_{1}\neq 0$. Consider the columns $\boldsymbol{\alpha}^{(1)}\odot\mathbf{f}^{(1)},\boldsymbol{\alpha}^{(2)}\odot\mathbf{f}^{(1)},\dots,\boldsymbol{\alpha}^{(\rho)}\odot\mathbf{f}^{(1)}$ in the matrix $\mathbf{\Phi}_{w}$, and their corresponding elements $u_{1},u_{L+1},\dots,u_{(\rho-1)L+1}$ in the vector $\mathbf{u}$. Let $\tilde{u}_{k}\triangleq u_{(k-1)L+1}$ for $k\in[\rho]$, and let $\tilde{\mathbf{u}}\triangleq(\tilde{u}_{1},\dots,\tilde{u}_{\rho})$. Note that $\tilde{\mathbf{u}}\neq 0$ (by construction). Consider the vector

[TABLE]

Expanding $\mathbf{g}=(g_{1},\dots,g_{w})^{\mathsf{T}}$, we get $g_{i}=(\tilde{u}_{1}\alpha_{i}^{0}+\tilde{u}_{2}\alpha_{i}^{1}+\dots+\tilde{u}_{\rho}\alpha_{i}^{\rho-1})f_{i,1}$ for all $i\in[w]$. Note that there exists $i\in[w]$ such that the coefficient of $f_{i,1}$ in $g_{i}$, i.e., $\tilde{u}_{1}\alpha_{i}^{0}+\tilde{u}_{2}\alpha_{i}^{1}+\dots+\tilde{u}_{\rho}\alpha_{i}^{\rho-1}$, is non-zero. The proof is by the way of contradiction. Suppose that for all $i\in[w]$ the coefficient of $f_{i,1}$ in $g_{i}$ is zero. Let $\mathbf{M}\triangleq((\alpha_{i}^{k-1})_{i\in[w],k\in[\rho]})$. Then it is easy to see that $\mathbf{M}\cdot\tilde{\mathbf{u}}=0$. Since $\mathbf{M}$ is a $w\times\rho$ Vandermonde matrix with $\rho<w$, then $\mathrm{rank}(\mathbf{M})=\rho$. This implies that $\tilde{\mathbf{u}}=0$. This is however a contradiction because $\tilde{\mathbf{u}}\neq 0$ (by assumption). Thus, for some $i\in[w]$ the coefficient of $f_{i,1}$ in $g_{i}$ must be non-zero. Thus, every matrix in $\mathcal{F}_{w}$ for $w>\rho$ contains at least one entry which can be written as a linear combination of the rest of the entries. Under the GRE model, this readily implies that $\mathcal{F}_{w}$ is a set of measure zero.

Noting that $\mathcal{F}=\cup_{w=1}^{t}\mathcal{F}_{w}$ and taking a union bound over all $w$ ($1\leq w\leq t$), it follows that for $t>L$, $\mathcal{F}$ is a set of measure zero. This completes the proof. ∎

VII-B Probability of Undetected Error

Similarly as in the case of the finite fields, both the MSSR decoding algorithm and the collaborative Peterson’s decoding algorithm give an error locator polynomial $\Lambda(z)$ over the real field ($\mathbb{R}$) of the least possible degree which satisfies all the syndrome equations in (5). This implies that these decoding algorithms have the ML certificate property (for details, see Section VI-B).

As was shown by Dutta et al. in [3, Theorem 3], under the GRE model, when the number of errors (i.e., the Hamming weight of the error matrix) is less than $N-K$, with probability $1$ the closest codeword to the received vector is the transmitted codeword. This implies that for any decoding algorithm satisfying the ML certificate property, the set of all bad error matrices (defined in Section VI-B) is of measure zero, and thereby, the probability of undetected error is zero.

Theorem 10**.**

Under the GRE model, for all $t\leq N-K-1$ (and in particular, for all $t\leq t_{\max}=\frac{L}{L+1}(N-K)$), $P_{\mathrm{ML}}(t)=0$.

VIII Numerical Results

We present simulation results for $N=8$, $K=2$, and $\alpha_{i}=0.9^{i}$ for different $L$. Fig. 1 shows the probability of error ($P_{e}(t)=P_{F}(t)+P_{\rm ML}(t)$) for decoding RS codes individually using Peterson’s algorithm ($L=1$), decoding RS codes individually using the $\ell_{1}$ minimization decoder, and collaborative decoding using the CPDA algorithm with $L=6$. For each data point, 12500 IRS codewords were simulated. It can be seen that the CPDA with $L=6$ corrects all $t$ errors for $t\leq N-K-1$, which is a significant improvement over decoding RS codes individually. This is consistent with the theoretical results. The probability of error for the $\ell_{1}$ minimization decoder remains fairly high for several values of $t\leq N-K-1$. These results are consistent with the results of Candes and Tao (Figures 2 and 3 in [14]). This shows that individually decoding RS decoder using the $\ell_{1}$-minimization decoder does not suffice to achieve small probability of error as suggested in [3]; whereas, collaborative decoding can achieve the decoding radius bound of $N-K-1$ with polynomial complexity.

For larger values of $N$ and $K$, we noticed that computing the rank of $\mathbf{S}_{L}(t)$ had numerical inaccuracies. This is a well-known issue with decoding RS codes over the real field. Interestingly, from simulations, we observe that collaborative decoding seems to alleviate this issue. Table I shows the probability of error ($P_{e}(t)=P_{F}(t)+P_{\rm ML}(t)$) for $N=20$, $K=12$ and $\alpha_{i}=i$. For a fixed number of errors, increasing $L$ improved the condition number of $\mathbf{S}_{L}(t)^{\sf T}\mathbf{S}_{L}(t)$. With $L=20$, we were able to decode up to $N-K-1$ errors with $P_{e}(t)=0$ in 12500 trials.

Our results have shown that collaborative decoding of Polynomial codes can correct up to $t_{\max}=\frac{L}{L+1}(N-K)$ errors. It can be seen that $t_{\rm max}=N-K-1$ for all $L\geq N-K-1$ and hence, it is natural to wonder if there is any advantage in increasing $L$ beyond $N-K-1$. Here we empirically show that increasing $L$ improves the numerical stability of the collaborative Peterson’s algorithm for determining the error locator polynomial. Fig. 2 ($N=8$, $K=2$, $\alpha_{i}=0.9^{i}$) shows a plot of the average condition number of the stacked syndome matrix $\mathbf{S}_{L}(t)$ (defined in (5)) as a function of $t$ for different $L$. It can be seen from simulations that for all $t$, increasing $L$ decreases the average condition number. Since the collaborative Peterson’s algorithm requires inversion of the matrix $\mathbf{S}_{L}^{\mathsf{T}}(t)\mathbf{S}_{L}(t)$, the numerical stability of the algorithm will improve with increasing $L$.