Universal secure rank-metric coding schemes with optimal communication overheads

TL;DR

This paper introduces a method to transform rank-metric coding schemes to reduce communication overheads in secure network coding and storage systems, achieving optimal rates and universal efficiency.

Contribution

It presents a novel transformation technique for linear rank-metric codes that lowers communication overheads while maintaining security and error correction capabilities.

Findings

Achieves universally optimal communication overheads for n ≤ m

Transforms Gabidulin codes to improve efficiency

Applicable to other rank distance codes when n > m

Abstract

We study the problem of reducing the communication overhead from a noisy wire-tap channel or storage system where data is encoded as a matrix, when more columns (or their linear combinations) are available. We present its applications to reducing communication overheads in universal secure linear network coding and secure distributed storage with crisscross errors and erasures and in the presence of a wire-tapper. Our main contribution is a method to transform coding schemes based on linear rank-metric codes, with certain properties, to schemes with lower communication overheads. By applying this method to pairs of Gabidulin codes, we obtain coding schemes with optimal information rate with respect to their security and rank error correction capability, and with universally optimal communication overheads, when $ n \leq m $, being $ n $ and $ m $ the number of columns and number of rows, respectively. Moreover, our method can be applied to other families of maximum rank distance codes when $ n > m $. The downside of the method is generally expanding the packet length, but some practical instances come at no cost.

Equations

$$\begin{array}{rl}{\mathcal{Y}}_X=\{Y\in {\mathbb{F}}_q^{\alpha m\times N}\mid & Y=X{A}^T+E,\\ & E\in {\mathbb{F}}_q^{\alpha m\times N},\mathrm{Rk}(E)\le t\},\\ {\mathcal{W}}_X=\{W\in {\mathbb{F}}_q^{\alpha m\times \mu }\mid & W=X{B}^T,B\in {\mathbb{F}}_q^{\mu \times n}\},\end{array}$$

$$F:{\mathbb{F}}_q^{\alpha m\times \ell }⟶{\mathbb{F}}_q^{\alpha m\times n},$$

$$R=\frac{{\log }_q(\#\mathcal{S})}{{\log }_q(\#\mathcal{X})}=\frac{\alpha m\ell }{\alpha mn}=\frac{\ell }{n}.$$

$$D_A(Y)=S,$$

$$H(S|W)=H(S),$$

$${\phi }_n:{\mathbb{F}}_{q^m}^{\alpha \times n}⟶{\mathbb{F}}_q^{\alpha m\times n}$$

$$c_{i,j}=\sum _{u=1}^m{d}_{(i-1)m+u,j}{\gamma }_u,$$

$${\phi }_N\left(C{A}^T+E\right)={\phi }_n(C)A^T+{\phi }_N(E),$$

$${\tilde{c}}_{i,j}=\sum _{v=1}^n{c}_{i,v}{a}_{j,v}=\sum _{v=1}^n\left(\sum _{u=1}^m{d}_{(i-1)m+u,v}{\gamma }_u\right)a_{j,v}$$

$$=\sum _{u=1}^m\left(\sum _{v=1}^n{d}_{(i-1)m+u,v}{a}_{j,v}\right){\gamma }_u,$$

$${\tilde{c}}_{i,j}=\sum _{u=1}^m{\tilde{d}}_{(i-1)m+u,j}{\gamma }_u,$$

$${\tilde{d}}_{(i-1)m+u,j}=\sum _{v=1}^n{d}_{(i-1)m+u,v}{a}_{j,v},$$

$$d_R\left({\mathcal{C}}_1,{\mathcal{C}}_2\right)=\min \left\{{\mathrm{Rk}}_q(\mathbf{e})\mid \mathbf{e}\in {\mathcal{C}}_1,\mathbf{e}\notin {\mathcal{C}}_2\right\}.$$

$$D_A\left({\left(E_{A,i}\left(C{\mathbf{a}}_i^T+{\mathbf{e}}_i\right)\right)}_{i=1}^d\right)=S,$$

$$\mathrm{DB}(A)=\frac{{\sum }_{i=1}^d{\beta }_{i}m}{\alpha m}=\sum _{i=1}^d\frac{{\beta }_i}{\alpha }\text{and}\mathrm{CO}(A)=\sum _{i=1}^d\frac{{\beta }_i}{\alpha }-\ell .$$

$$\ell \le n-2t-\rho -\mu .$$

$$\mathrm{CO}(A)\ge \frac{\ell (2t+\mu )}{d-2t-\mu },$$

$${\left(E_{A,i}\left(C_1{\mathbf{a}}_i^T\right)\right)}_{i=1}^{d-2t}={\left(E_{A,i}\left(C_2{\mathbf{a}}_i^T\right)\right)}_{i=1}^{d-2t}.$$

$$\sum _{i=1}^{d-2t}\frac{{\beta }_i}{\alpha }\ge \frac{\ell (d-2t)}{d-2t-\mu },$$

$$\frac{{\beta }_{d-2t-\mu }}{\alpha }\ge \frac{\ell }{d-2t-\mu }.$$

$$\mathrm{DB}(A)=\left(\sum _{i=1}^{d-2t}\frac{{\beta }_i}{\alpha }\right)+\left(\sum _{i=d-2t+1}^d\frac{{\beta }_i}{\alpha }\right)$$

$$\ge \frac{\ell (d-2t)}{d-2t-\mu }+2t\frac{\ell }{d-2t-\mu }=\frac{\ell d}{d-2t-\mu },$$

$${\mathcal{C}}_1={\mathcal{C}}^{(h)}⫋{\mathcal{C}}^{(h-1)}⫋\dots ⫋{\mathcal{C}}^{(2)}⫋{\mathcal{C}}^{(1)}\subseteq {\mathbb{F}}_{q^m}^n$$

$$d_R\left({\mathcal{C}}^{(j)},{\mathcal{C}}_2\right)\ge n-d_j+1,$$

$$\alpha =\mathrm{LCM}({\alpha }_1,{\alpha }_2,\dots ,{\alpha }_h).$$

$$\mathrm{CO}(A)=\frac{\ell \left(d-k^{(j)}+k_2\right)}{k^{(j)}-k_2},$$

$$G_{1}=\left(\begin{array}[]{c}G_{2}\\ G_{c}\end{array}\right)\in\mathbb{F}_{q^{m}}^{k_{1}\times n},$$

$$G^{(j)}=\left(\begin{array}[]{c}G^{(j+1)}\\ G^{(j+1)}_{c}\end{array}\right)\in\mathbb{F}_{q^{m}}^{k^{(j)}\times n},$$

$$p_{j}=\left\{\begin{array}[]{ll}\frac{\ell\alpha}{\alpha_{1}}&\textrm{if }j=1,\\ \frac{\ell\alpha}{\alpha_{j}}-\frac{\ell\alpha}{\alpha_{j-1}}&\textrm{if }1<j\leq h.\end{array}\right.$$

$$S=\left(\begin{array}[]{c}S_{1}\\ S_{2}\\ \vdots\\ S_{h}\end{array}\right),\quad R=\left(\begin{array}[]{c}R_{1}\\ R_{2}\\ \vdots\\ R_{h}\end{array}\right),$$

Full text

Keywords: Communication overheads, crisscross error-correction, decoding bandwidth, information-theoretical security, rank-metric codes.

MSC: 94A60, 94A62, 94B99.

1 Introduction

Universal secure linear network coding with errors and erasures was first studied in [22], where rank-metric coding schemes were proposed to protect messages sent over a linearly coded network from link errors, erasures and information leakage to a wire-tapper. Similarly, rank-metric codes have been applied to storage systems where data is stored as a matrix and where errors and erasures affect several rows and/or columns, also called crisscross errors and erasures [21]. These errors and erasures have been recently motivated by correlated and mixed failures in distributed storage systems where data is stored in several data centers (columns), which in turn store several blocks of data (rows). See [14].

In this paper, we study how to reduce the communication overhead from such a noisy wire-tap channel or storage system to the receiver, when more columns, or their linear combinations, are available: Less ingoing links to the receiver fail in the network case, or more data centers are available and contacted in the distributed storage case. As it has been noticed in secret sharing in the literature [2, 13, 24], which corresponds to Hamming-metric erasure-correction and security, if more pieces of data (columns in our case) are available, they can be preprocessed via subpacketization so that the overall transmitted information from the channel or storage system to the receiver is reduced.

A similar concept of subpacketization has been recently developed for Reed-Solomon codes in [10]. In another direction, coding schemes recovering part of the encoded data (a node in a storage system, for instance), with respect to the Hamming metric, have already been studied, giving rise to regenerating codes [5, 6, 20], which reduce communication bandwidth, and locally repairable codes [9, 12, 23], which reduce the number of contacted nodes. The latter codes have been recently extended to the rank metric in [14]. In contrast, our aim is to recover the whole uncoded data while reducing the communication bandwidth, as in [2, 13, 24], but with respect to the rank metric and, as a consequence, with respect to the crisscross metric.

We illustrate and motivate the problem with a pair of examples. The details of the constructions will be given in Subsection 5.1.

Example 1**.**

Consider a linearly coded network, as in [22, Sec. VII-A], over a finite field of size $q=256$ ($8$-bit symbols), with packet length $m=2048$, number of outgoing links from the source $n=40$, at least $N\geq n$ ingoing links to the sink, and where $\mu\leq 8$ links may be wire-tapped and $\rho\leq 16$ ingoing links to the sink may fail.

In [22, Th. 11], a coding scheme is given with optimal information rate $16/40$, able to correct the given number of erasures and secure under the given number of observations over such network, independently of its inner code (universally). The overall communication overhead from the last ingoing links to the sink is of $8$ packets: The source wants to transmit $16$ uncoded packets and the sink receives $24$ encoded packets.

Thanks to Theorem 2 and dividing each packet into $32$ subpackets of length $64$ each, we will obtain a coding scheme with the same parameters, but such that the overall communication overhead at the ingoing links to the sink is of $4$ packets (the minimum possible) if none of them fail (only $20$ packets are received by the sink).

Example 2**.**

Let again $n=40$ and $m=2048$, and consider a distributed storage system where data is stored as an $m\times n$ matrix over the same finite field ($q=256$), where each column corresponds to a data center that stores $n$ symbols over $\mathbb{F}_{q}$, that is, $40$ $8$-bit symbols. Assume that $\rho$ data centers may fail or not be available, errors occur along $t$ rows and/or columns due to certain correlations, and a wire-tapper eavesdrops $\mu$ data centers. Assume also that $\rho+2t\leq 16$ and $\mu\leq 8$.

As in the previous example, the use of a pair of maximum rank distance codes allows to obtain the desired reliability and security while achieving the optimal information rate $16/40$ (see [21]), with a communication overhead of $8$ packets from the contacted data centers to the receiver. Again, in this work we obtain a coding scheme with the same parameters but where the communication overhead is reduced to $4$ packets (the minimum possible) if no errors occur and all data centers are available and contacted.

The paper is organized as follows: In Section 2, we establish the information-theoretical setting, defining coherent linearized noisy wire-tap channels, which we take from [22], and we establish a method of subpacketization that allows to use linear codes over the extension field. In Section 3, we define communication overheads for these linearized channels and give lower bounds on these parameters similar to those in [13]. In Section 4, we give the main contribution of this paper, which is a general method to transform coding schemes based on pairs of linear rank-metric codes, with certain properties, into coding schemes with lower communication overheads. In Section 5, we apply Gabidulin codes [8, 21] to obtain coding schemes with optimal information rates and communication overheads for $n\leq m$, which can be seen as a rank-metric analog of the constructions in [2, 13]. However, our method allows us to correct errors, and not only erasures as in the secret sharing case [2, 13], and can be applied to other families of maximum rank distance codes, such as those in [7] for $n>m$. Finally, in Section 6, we discuss the applications in universal secure linear network coding and secure distributed storage with crisscross errors and erasures.

Notation

Throughout the paper, we fix a prime power $q$ and positive integers $m$, $n$, $N$, $\alpha$, $\ell$, $t$, $\rho$ and $\mu$. We denote by $\mathbb{F}_{q}$ the finite field with $q$ elements, $\mathbb{F}_{q}^{n}$ denotes the set of row vectors of length $n$ over $\mathbb{F}_{q}$, and $\mathbb{F}_{q}^{m\times n}$ denotes the set of $m\times n$ matrices over $\mathbb{F}_{q}$. In this paper, a code is a subset of either $\mathbb{F}_{q}^{n}$ or $\mathbb{F}_{q}^{m\times n}$, whose linearity properties are specified in each case. For an $\mathbb{F}_{q^{m}}$-linear code $\mathcal{C}\subseteq\mathbb{F}_{q^{m}}^{n}$, we will denote by $\mathcal{C}^{\perp}$ its dual code with respect to the usual $\mathbb{F}_{q^{m}}$-bilinear inner product. We also use the notation $[n]=\{1,2,\ldots,n\}$ and $[m,n]=\{m,m+1,\ldots,n\}$ whenever $m\leq n$, and we denote by $H(X)$, $H(X\mid Y)$ and $I(X;Y)$ the entropy, conditional entropy and mutual information of the random variables $X$ and $Y$, respectively (see [3]), where logarithms will always be taken with base $q$.

2 Information-theoretical setting and preliminaries

2.1 Coherent linearized channels and coset coding schemes

We will consider the secret message $S$ to be a uniform random variable in $\mathcal{S}=\mathbb{F}_{q}^{\alpha m\times\ell}$, and we will consider noisy wire-tap channels (which can also be thought of as distributed storage systems) as given in [22]:

Definition 1** (Coherent linearized channel [22]).**

We define a coherent linearized noisy wire-tap channel with $t$ errors, $\rho$ erasures with erasure matrix $A\in\mathbb{F}_{q}^{N\times n}$ of rank at least $n-\rho$, and $\mu$ observations as a channel with input a variable $X\in\mathcal{X}=\mathbb{F}_{q}^{\alpha m\times n}$, output to the receiver $Y\in\mathcal{Y}=\mathbb{F}_{q}^{\alpha m\times N}$, and output to the eavesdropper $W\in\mathcal{W}=\mathbb{F}_{q}^{\alpha m\times\mu}$, together with a conditional probability distribution $P(Y,W|X)$ such that

[TABLE]

where $\mathcal{Y}_{X}=\{Y\in\mathcal{Y}\mid P(Y|X)>0\}$ and $\mathcal{W}_{X}=\{W\in\mathcal{W}\mid P(W|X)>0\}$, for a given $X\in\mathcal{X}$.

In [22], it is shown that a linearly coded network over $\mathbb{F}_{q}$ with link errors, erasures and information leakage, and where the last coding coefficients are known to the receiver, can be modelled as a coherent linearized noisy wire-tap channel. We will focus on this scenario and discuss how to translate the results to the distributed storage scenario with crisscross errors and erasures in Subsection 6.2, since the latter can be seen as a simpler case.

As encoders, we consider coset coding schemes as in [16, Def. 7], which are a particular case of those in [22].

Definition 2** (Coset coding schemes [16]).**

A coset coding scheme over the field $\mathbb{F}_{q}$ with secret message set $\mathcal{S}=\mathbb{F}_{q}^{\alpha m\times\ell}$ and coded message set $\mathcal{X}=\mathbb{F}_{q}^{\alpha m\times n}$ is a randomized function

[TABLE]

where, for every $S\in\mathbb{F}_{q}^{\alpha m\times\ell}$, $C=F(S)$ is the uniform random variable over a set $\mathcal{C}_{S}\subseteq\mathbb{F}_{q}^{\alpha m\times n}$. To allow correct decoding, we also assume that $\mathcal{C}_{S}\cap\mathcal{C}_{T}=\varnothing$ if $S\neq T$. Finally, we define the information rate of the scheme as

[TABLE]

In linear network coding, universal reliability and security means correcting a number of link errors and erasures and being secure under a number of link observations, independently of the network inner code. This leads in [22] to the following definition:

Definition 3** (Universal schemes [22]).**

We say that the coset coding scheme $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$ is:

Universally $t$-error and $\rho$-erasure-correcting if, for every coherent linearized channel with $t$ errors, $\rho$ erasures and erasure matrix $A\in\mathbb{F}_{q}^{N\times n}$, there exists a decoding function $D_{A}:\mathcal{Y}\longrightarrow\mathcal{S}$ such that

[TABLE]

for all $Y\in\bigcup_{X\in\mathcal{C}_{S}}\mathcal{Y}_{X}$ and all $S\in\mathcal{S}$. 2. 2.

Universally secure under $\mu$ observations if, for every coherent linearized channel with $\mu$ observations, it holds that

[TABLE]

or equivalently $I(S;W)=0$, for all $W\in\bigcup_{X\in\mathcal{C}_{S}}\mathcal{W}_{X}$ and all $S\in\mathcal{S}$.

2.2 Using linear codes over the extension field

In what follows, we will make use of codes that are linear over the extension field $\mathbb{F}_{q^{m}}$. To that end, we need to see how to identify matrices in $\mathbb{F}_{q}^{\alpha m\times n}$ with matrices in $\mathbb{F}_{q^{m}}^{\alpha\times n}$:

Definition 4**.**

Fix a basis $\gamma_{1},\gamma_{2},\ldots,\gamma_{m}$ of $\mathbb{F}_{q^{m}}$ as a vector space over $\mathbb{F}_{q}$, and define the map

[TABLE]

as follows: Given a matrix $C\in\mathbb{F}_{q^{m}}^{\alpha\times n}$ with entries $c_{i,j}\in\mathbb{F}_{q^{m}}$, for $i=1,2,\ldots,\alpha$ and $j=1,2,\ldots,n$, we define $\varphi_{n}(C)$ as the unique $\alpha m\times n$ matrix with coefficients $d_{l,j}\in\mathbb{F}_{q}$, for $l=1,2,\ldots,\alpha m$ and $j=1,2,\ldots,n$, such that

[TABLE]

for $i=1,2,\ldots,\alpha$ and $j=1,2,\ldots,n$. Finally, we define the rank over $\mathbb{F}_{q}$ of a matrix $E\in\mathbb{F}_{q^{m}}^{\alpha\times n}$ as the rank over $\mathbb{F}_{q}$ of the matrix $\varphi_{n}(E)\in\mathbb{F}_{q}^{\alpha m\times n}$, and we denote it by ${\rm Rk}_{q}(E)$.

The key result is that the effect of coherent linearized noisy wire-tap channels in Definition 1 remains unchanged by the map $\varphi_{n}$, as we will now see:

Lemma 1**.**

Let $C\in\mathbb{F}_{q^{m}}^{\alpha\times n}$, $A\in\mathbb{F}_{q}^{N\times n}$ and $E\in\mathbb{F}_{q^{m}}^{\alpha\times N}$. It holds that

[TABLE]

and ${\rm Rk}_{q}(E)={\rm Rk}(\varphi_{N}(E))$ by definition.

Proof.

The additive property of $\varphi_{n}$ is clear from the definition, so we may assume that $E=0$. Denote the entries of $C$ and $\varphi_{n}(C)$ as in Definition 4, and let $a_{v,j}$, $\widetilde{c}_{i,j}$ and $\widetilde{d}_{l,j}$ be the entries of $A$, $CA^{T}$ and $\varphi_{N}\left(CA^{T}\right)$, respectively, for $v=1,2,\ldots,N$, $i=1,2,\ldots,\alpha$, $l=1,2,\ldots,\alpha m$ and $j=1,2,\ldots,n$. It holds that

[TABLE]

[TABLE]

but it also holds that

[TABLE]

for $i=1,2,\ldots,\alpha$ and $j=1,2,\ldots,n$. Since $\gamma_{1},\gamma_{2},\ldots,\gamma_{m}$ is a basis of $\mathbb{F}_{q^{m}}$ over $\mathbb{F}_{q}$ and $\sum_{v=1}^{n}d_{(i-1)m+u,v}a_{j,v}\in\mathbb{F}_{q}$, for $i=1,2,\ldots,\alpha$, and $j=1,2,\ldots,n$, we conclude that

[TABLE]

for $i=1,2,\ldots,\alpha$, $u=1,2,\ldots,m$ and $j=1,2,\ldots,n$, which means that $\varphi_{N}\left(CA^{T}\right)=\varphi_{n}(C)A^{T}$, and the result follows. ∎

Hence we may identify the sets $\mathbb{F}_{q}^{\alpha m\times n}$ and $\mathbb{F}_{q^{m}}^{\alpha\times n}$, seen as $\mathbb{F}_{q}$-linear vector spaces together with the metric given by the rank and the function ${\rm Rk}_{q}$, respectively. We will do this repeatedly throughout the paper.

To conclude the section, we recall the construction of coset coding schemes in [16, Def. 4] based on pairs of $\mathbb{F}_{q^{m}}$-linear codes with $\alpha=1$ (no subpacketization).

Definition 5** (Nested coset coding schemes [16]).**

A nested coset coding scheme (with $\alpha=1$) is a coset coding scheme such that $\mathcal{C}_{S}=\varphi(S)+\mathcal{C}_{2}$, where $\mathcal{C}_{2}\subsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$ are $\mathbb{F}_{q^{m}}$-linear codes and $\varphi:\mathbb{F}_{q^{m}}^{\ell}\longrightarrow\mathcal{W}$ is a vector space isomorphism over $\mathbb{F}_{q^{m}}$, for an $\mathbb{F}_{q^{m}}$-linear space $\mathcal{W}\subseteq\mathbb{F}_{q^{m}}^{n}$ such that $\mathcal{C}_{1}=\mathcal{C}_{2}\oplus\mathcal{W}$, where $\oplus$ denotes the direct sum of vector spaces.

To measure the reliability and security of these coding schemes, we need the concept of relative minimum rank distance, which is a particular case of [16, Def. 2]:

Definition 6** (Relative minimum rank distance [16]).**

Given $\mathbb{F}_{q^{m}}$-linear codes $\mathcal{C}_{2}\subsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$, we define their relative minimum rank distance as

[TABLE]

The minimum rank distance of a single code $\mathcal{C}\subseteq\mathbb{F}_{q^{m}}^{n}$ is defined as $d_{R}(\mathcal{C})=d_{R}(\mathcal{C},\{\mathbf{0}\})$.

The next result, which follows directly from [16, Cor. 5 and Th. 4], gives the mentioned reliability and security performance of nested coset coding schemes. Recall that we denote by $\mathcal{C}^{\perp}$ the dual of an $\mathbb{F}_{q^{m}}$-linear code $\mathcal{C}\subseteq\mathbb{F}_{q^{m}}^{n}$ with respect to the usual $\mathbb{F}_{q^{m}}$-bilinear inner product in $\mathbb{F}_{q^{m}}^{n}$.

Lemma 2** ([16]).**

Given $\mathbb{F}_{q^{m}}$-linear codes $\mathcal{C}_{2}\subsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$, the nested coset coding scheme in Definition 5 is universally $t$-error and $\rho$-erasure-correcting if, and only if, $2t+\rho<d_{R}(\mathcal{C}_{1},\mathcal{C}_{2})$, and is universally secure under $\mu$ observations if, and only if, $\mu<d_{R}\left(\mathcal{C}_{2}^{\perp},\mathcal{C}_{1}^{\perp}\right)$.

Observe that $d_{R}(\mathcal{C}_{1})\leq d_{R}(\mathcal{C}_{1},\mathcal{C}_{2})$ and $d_{R}\left(\mathcal{C}_{2}^{\perp}\right)\leq d_{R}\left(\mathcal{C}_{2}^{\perp},\mathcal{C}_{1}^{\perp}\right)$, hence the minimum rank distances of $\mathcal{C}_{1}$ and $\mathcal{C}_{2}^{\perp}$ give sufficient conditions on the number of correctable errors and erasures and on the number of links that may be wire-tapped without information leakage, respectively.

3 Communication overheads in coherent linearized channels

In this section we formalize how, as in communication efficient secret sharing [2, 13, 24], if a coset coding scheme is able to correct $t$ errors and $\rho$ erasures, but $d>n-\rho$ pieces of information are available (the rank of $A$ is at least $d$), then we may reduce the communication overhead from the channel to the receiver by making use of the additional $d-n+\rho>0$ linearly independent rows of $A$. Observe that only erasures, and not errors, are considered in the Hamming analog described in [2, 13, 24].

Let $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$ be a coset coding scheme, let $A\in\mathbb{F}_{q}^{d\times n}$ be of rank $d$ (if $A\in\mathbb{F}_{q}^{N\times n}$ has rank $d<N$, we may delete or ignore linearly dependent rows), let $\mathbf{a}_{1},\mathbf{a}_{2},\ldots,\mathbf{a}_{d}\in\mathbb{F}_{q}^{n}$ be its rows, and let $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$ be preprocessing functions, where $1\leq\beta_{i}\leq\alpha$, for $i=1,2,\ldots,d$. We define their correction capability with respect to $F$ as follows:

Definition 7**.**

For a full-rank matrix $A\in\mathbb{F}_{q}^{d\times n}$, the preprocessing functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d$, are $t$-error-correcting with respect to the coset coding scheme $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$ if there exists a decoding function $D_{A}:\prod_{i=1}^{d}\mathbb{F}_{q}^{\beta_{i}m}\longrightarrow\mathbb{F}_{q}^{\alpha m\times\ell}$ such that

[TABLE]

for all $C\in\mathcal{C}_{S}$, all $S\in\mathbb{F}_{q}^{\alpha m\times\ell}$ and all error matrices $E\in\mathbb{F}_{q}^{\alpha m\times d}$ of rank at most $t$ with columns $\mathbf{e}_{1},\mathbf{e}_{2},\ldots,\mathbf{e}_{d}\in\mathbb{F}_{q}^{\alpha m}$.

We define then the decoding bandwidth and communication overhead as $q$-analogs of those in [13, Def. 2]:

Definition 8** (Decoding bandwidth and communication overhead).**

For a full-rank matrix $A\in\mathbb{F}_{q}^{d\times n}$ and functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d$, we define their decoding bandwidth and communication overhead, respectively, as

[TABLE]

Thus, if a packet is a vector in $\mathbb{F}_{q}^{\alpha m}$, then the decoding bandwidth is the amount (which need not be an integer due to the subpacketization) of packets that the receiver obtains, or needs to obtain, from the channel, and the communication overhead is the difference with respect to the original number of uncoded packets.

Observe that, fixing $n$ and $\ell$ (thus the information rate), we may only focus on communication overheads, since both behave equally.

To measure the quality of a coset coding scheme, we need the following two bounds. The first is given in [22, Th. 12] and can be seen as a $q$-analog of the bound in [13, Prop. 1], although considering also errors and not only erasures:

Proposition 1** ([22]).**

If the coset coding scheme $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$ is universally $t$-error and $\rho$-erasure-correcting, and universally secure under $\mu$ observations, then

[TABLE]

Next we give a $q$-analog of the bound in [13, Th. 1], again adding the effect of errors, which was not considered in [13]:

Proposition 2**.**

If the coset coding scheme $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$ is universally secure under $\mu$ observations, then for a full-rank matrix $A\in\mathbb{F}_{q}^{d\times n}$ and preprocessing functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d$, that are $t$-error-correcting with respect to $F$, it holds that:

[TABLE]

Proof.

We may assume without loss of generality that $\beta_{1}\leq\beta_{2}\leq\ldots\leq\beta_{d}$ as in the proof of [13, Th. 1].

First, we prove that the preprocessing functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d-2t$ are [math]-error-correcting with respect to $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$. If they were not, then there would exist $C_{1}\in\mathcal{C}_{S_{1}}$ and $C_{2}\in\mathcal{C}_{S_{2}}$, with $S_{1}\neq S_{2}$, such that

[TABLE]

On the other hand, there exist $\mathbf{e}_{i}\in\mathbb{F}_{q}^{\alpha m}$ such that $C_{1}\mathbf{a}_{i}^{T}+\mathbf{e}_{i}=C_{2}\mathbf{a}_{i}^{T}$, for $i=d-2t+1,d-2t+2,\ldots,d-t$, and $C_{1}\mathbf{a}_{i}^{T}=C_{2}\mathbf{a}_{i}^{T}+\mathbf{e}_{i}$, for $i=d-t+1,d-t+2,\ldots,d$. Thus we see that the preprocessing functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d$, cannot be $t$-error-correcting with respect to $F$, which is a contradiction.

Next, defining $f_{i}=E_{A,i}\left(C\mathbf{a}_{i}^{T}\right)$, where $C=F(S)$, for $i=1,2,\ldots,d-2t$ and $S\in\mathbb{F}_{q}^{\alpha m\times\ell}$, we may prove exactly as in the proof of [13, Th. 1] that

[TABLE]

and also

[TABLE]

Now using that $\beta_{d-2t-\mu}\leq\beta_{d-2t-\mu+1}\leq\ldots\leq\beta_{d}$, and combining Equations (6) and (7), we conclude that

[TABLE]

[TABLE]

and the bound on ${\rm CO}(A)$ follows by substracting $\ell$ to this inequality. ∎

4 A general construction based on linear rank-metric codes

In this section, given a nested coset coding scheme (Definition 5) able to correct $t_{0}$ errors and $\rho_{0}$ erasures, for fixed positive integers $t_{0}$ and $\rho_{0}$, and given an arbitrary set $\mathcal{D}\subseteq[n-\rho_{0},n]$ such that $n-\rho_{0}\in\mathcal{D}$ (in particular for $\mathcal{D}=[n-\rho_{0},n]$), we construct a coset coding scheme able to correct $t_{0}$ errors and any $n-d$ erasures with lower communication overheads than the original scheme, for all $d\in\mathcal{D}$. Moreover, both the original scheme and the modified one are universally secure under the same number of observations. The downside of the method is multiplying the packet length of the original coset coding scheme by a parameter $\alpha$, depending on the involved codes, to achieve the desired subpacketization. The main result of the section is the following:

Theorem 1**.**

Take $\mathbb{F}_{q^{m}}$-linear codes $\mathcal{C}_{2}\subsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$, a positive integer $\rho_{0}$ such that $\rho_{0}<d_{R}(\mathcal{C}_{1},\mathcal{C}_{2})$, and choose any subset $\mathcal{D}\subseteq[n-\rho_{0},n]$ such that $n-\rho_{0}\in\mathcal{D}$. Denote $\mathcal{D}=\{d_{1},d_{2},\ldots,d_{h}\}$, where $d_{h}=n-\rho_{0}<d_{h-1}<\ldots<d_{2}<d_{1}$, and assume that there exists a sequence of nested $\mathbb{F}_{q^{m}}$-linear codes

[TABLE]

such that

[TABLE]

for $j=1,2,\ldots,h$. Define then $k_{1}=\dim(\mathcal{C}_{1})$, $k_{2}=\dim(\mathcal{C}_{2})$, $\ell=k_{1}-k_{2}$, $\alpha_{j}=k^{(j)}-k_{2}$ for $j=1,2,\ldots,h$, and

[TABLE]

There exists a coset coding scheme $F:\mathbb{F}_{q^{m}}^{\alpha\times\ell}\longrightarrow\mathbb{F}_{q^{m}}^{\alpha\times n}$ that is universally $t$-error and $\rho$-erasure-correcting if $2t+\rho<d_{R}\left(\mathcal{C}_{1},\mathcal{C}_{2}\right)$, and is universally secure under $\mu$ observations if $\mu<d_{R}\left(\mathcal{C}_{2}^{\perp},\mathcal{C}^{(1)\perp}\right)$.

In addition, for any $d\in\mathcal{D}$ and any full-rank matrix $A\in\mathbb{F}_{q}^{d\times n}$, there exist preprocessing functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d$, which are $t$-error-correcting with respect to $F$, whenever $2t<d_{R}\left(\mathcal{C}^{(j)},\mathcal{C}_{2}\right)-n+d$, and such that

[TABLE]

where $k^{(j)}=\dim\left(\mathcal{C}^{(j)}\right)$, for $j$ such that $d=d_{j}$.

4.1 Description of the construction for Theorem 1

Let the notation be as in Theorem 1 and take a generator matrix $G_{2}\in\mathbb{F}_{q^{m}}^{k_{2}\times n}$ of $\mathcal{C}_{2}$ and a generator matrix $G_{1}\in\mathbb{F}_{q^{m}}^{k_{1}\times n}$ of $\mathcal{C}_{1}$ of the form

[TABLE]

for some matrix $G_{c}\in\mathbb{F}_{q^{m}}^{\ell\times n}$. Decreasingly in $j=h-1,h-2,\ldots,2,1$, take a generator matrix $G^{(j)}\in\mathbb{F}_{q^{m}}^{k^{(j)}\times n}$ of $\mathcal{C}^{(j)}$ of the form

[TABLE]

for some matrix $G_{c}^{(j+1)}\in\mathbb{F}_{q^{m}}^{\left(k^{(j)}-k^{(j+1)}\right)\times n}$. Next define the following positive integers, which are analogous to the integers defined in [13, Eq. (11)]:

[TABLE]

Let $S\in\mathbb{F}_{q^{m}}^{\alpha\times\ell}$ be the secret message and generate uniformly at random a matrix $R\in\mathbb{F}_{q^{m}}^{\alpha\times k_{2}}$. Divide $S$ and $R$ as follows:

[TABLE]

where

[TABLE]

for $j=1,2,\ldots,h$. Next, we define the matrices

[TABLE]

where $M_{u}\in\mathbb{F}_{q^{m}}^{p_{u}\times k^{(1)}}$, and where the matrices $D_{u,v}\in\mathbb{F}_{q^{m}}^{p_{u}\times(\alpha_{h-v}-\alpha_{h-v+1})}$ are defined iteratively as follows: For $v=1,2,\ldots,h-1$, the components of the $v$-th column block

[TABLE]

are the components (after some fixed rearrangement) of

[TABLE]

whose size is $p_{h-v+1}\times\alpha_{h-v+1}$ (observe that $p_{j+1}\alpha_{j+1}=(\alpha_{j}-\alpha_{j+1})\ell\alpha/\alpha_{j}$). For convenience, we define the matrices

[TABLE]

for $j=1,2,\ldots,h$.

Finally, we define the coset coding scheme $F:\mathbb{F}_{q^{m}}^{\alpha\times\ell}\longrightarrow\mathbb{F}_{q^{m}}^{\alpha\times n}$ by

[TABLE]

To conclude, we define $E_{A,i}:\mathbb{F}_{q^{m}}^{\alpha}\longrightarrow\mathbb{F}_{q^{m}}^{\ell\alpha/\alpha_{j}}$ as follows. For $j=1,2,\ldots,h$, for $i=1,2,\ldots,d_{j}$, and for a full-rank matrix $A\in\mathbb{F}_{q}^{d_{j}\times n}$, we define $E_{A,i}(\mathbf{y}_{i})\in\mathbb{F}_{q^{m}}^{\ell\alpha/\alpha_{j}}$ by restricting $\mathbf{y}_{i}\in\mathbb{F}_{q^{m}}^{\alpha}$ to its first $\ell\alpha/\alpha_{j}$ rows.

4.2 Proof of Theorem 1

Let the notation be as in Theorem 1 and as in the previous subsection. We prove each statement in Theorem 1 separately:

1) The coset coding scheme is universally $t$-error and $\rho$-erasure-correcting if $2t+\rho<d_{R}(\mathcal{C}_{1},\mathcal{C}_{2})$: Take $A\in\mathbb{F}_{q}^{N\times n}$ of rank at least $n-\rho$ and an error matrix $E\in\mathbb{F}_{q^{m}}^{\alpha\times N}$ such that ${\rm Rk}_{q}(E)\leq t$. Divide $E$ in the same way as $S$ and $R$, that is,

[TABLE]

where $E_{j}\in\mathbb{F}_{q^{m}}^{p_{j}\times N}$, and observe that ${\rm Rk}_{q}(E_{j})\leq{\rm Rk}_{q}(E)\leq t$, for $j=1,2,\ldots,h$. From

[TABLE]

we obtain $S_{h}$ by Lemma 2, since ${\rm Rk}_{q}(E_{h})\leq t$ and $2t+\rho<d_{R}(\mathcal{C}_{1},\mathcal{C}_{2})$. By definition, we have obtained $D_{u,1}$, for $u=1,2,\ldots h-1$. Hence substracting $D_{h-1,1}G_{c}^{(h)}A^{T}$ from $(S_{h-1}|R_{h-1}|D_{h-1,1}|$ $0)G^{(1)}A^{T}+E_{h-1}$, we may obtain $(S_{h-1}|R_{h-1})G_{1}A^{T}+E_{h-1}$, and thus we obtain $S_{h-1}$ again by Lemma 2. Now, we have also obtained $D_{u,2}$, for $u=1,2,\ldots,h-2$. Proceeding iteratively in the same way, we see that we may obtain all the matrices $S_{j}$, for $j=1,2,\ldots,h$, and thus we obtain the whole message $S$.

2) The coset coding scheme is universally secure under any $\mu<d_{R}\left(\mathcal{C}_{2}^{\perp},\mathcal{C}_{1}^{\perp}\right)$ observations: We first need the following preliminary lemma, which follows from [18, Th. 3]:

Lemma 3**.**

Let $B\in\mathbb{F}_{q}^{\mu\times n}$ and let $\mathcal{C}_{2}\subsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$ be $\mathbb{F}_{q^{m}}$-linear codes. If ${\rm Rk}(B)<d_{R}\left(\mathcal{C}_{2}^{\perp},\mathcal{C}_{1}^{\perp}\right)$, then

[TABLE]

where $\mathcal{C}B^{T}=\left\{\mathbf{c}B^{T}\mid\mathbf{c}\in\mathcal{C}\right\}\subseteq\mathbb{F}_{q^{m}}^{\mu}$, for a code $\mathcal{C}\subseteq\mathbb{F}_{q^{m}}^{n}$.

Proof.

See Appendix A. ∎

Take $B\in\mathbb{F}_{q}^{\mu\times n}$, and assume that the eavesdropper obtains

[TABLE]

The random variable $W$ has support inside the $\mathbb{F}_{q^{m}}$-linear vector space

[TABLE]

Recall from [3, Th. 2.6.4] that, if a random variable $X$ has support in the set $\mathcal{X}$, then $H(X)\leq\log_{q}(\#\mathcal{X})$. Hence

[TABLE]

where dimensions are taken over $\mathbb{F}_{q^{m}}$. On the other hand, using the analogous notation $\mathcal{C}_{2B,\alpha}$ for $\mathcal{C}_{2}$ instead of $\mathcal{C}^{(1)}$, it holds that

[TABLE]

since, given a value of $S$, the variable $W$ is a uniform random variable over an $\mathbb{F}_{q^{m}}$-linear affine space obtained by translating the vector space $\mathcal{C}_{2B,\alpha}$. Hence we obtain that

[TABLE]

[TABLE]

where the last equality follows from Lemma 3. Thus $I(S;W)=0$ and we are done.

3) The preprocessing functions are $t$-error-correcting for any $2t<d_{R}\left(\mathcal{C}^{(j)},\mathcal{C}_{2}\right)-n+d$, where $d=d_{j}$: Fix $d\in\mathcal{D}$ and a full-rank matrix $A\in\mathbb{F}_{q}^{d\times n}$, and let $E_{A,i}:\mathbb{F}_{q^{m}}^{\alpha}\longrightarrow\mathbb{F}_{q^{m}}^{\ell\alpha/\alpha_{j}}$ be preprocessing functions as in the previous subsection, for $i=1,2,\ldots,d$, and where $j$ is such that $d=d_{j}$.

Let $E\in\mathbb{F}_{q^{m}}^{\alpha\times d}$ be an error matrix such that ${\rm Rk}_{q}(E)\leq t$, and let $\mathbf{e}_{1},\mathbf{e}_{2},\ldots,\mathbf{e}_{d}\in\mathbb{F}_{q^{m}}^{\alpha}$ be its columns. By definition, $E_{A,i}\left(C\mathbf{a}_{i}^{T}+\mathbf{e}_{i}\right)$ is the $i$-th column of

[TABLE]

for $i=1,2,\ldots,d$, and a submatrix $E^{\prime}_{j}\in\mathbb{F}_{q^{m}}^{\ell\alpha/\alpha_{j}\times d}$ of $E$, which thus satisfies that ${\rm Rk}_{q}(E_{j}^{\prime})\leq{\rm Rk}_{q}(E)\leq t$. Therefore, we may obtain the matrix $M_{j}^{\prime}$ as in item 1, since $2t+n-d<d_{R}(\mathcal{C}^{(j)},\mathcal{C}_{2})$. By definition, the matrices $S_{1},S_{2},\ldots,S_{j}$ are contained in $M_{j}^{\prime}$. Moreover, the matrices $D_{1,h-j},D_{2,h-j},\ldots,D_{j,h-j}$ are also contained in $M_{j}^{\prime}$, and from them we obtain by definition $S_{j+1}$ and $D_{j+1,1},D_{j+1,2},\ldots,D_{j+1,h-j-1}$. Now, the matrices $D_{1,h-j-1},D_{2,h-j-1},\ldots,D_{j,h-j-1}$ are contained in $M_{j}^{\prime}$ and we also have $D_{j+1,h-j-1}$, hence we may obtain by definition $S_{j+2}$ and $D_{j+2,1},D_{j+2,2},\ldots,D_{j+2,h-j-2}$. Continuing iteratively in this way, we may obtain all $S_{1},S_{2},\ldots,S_{h}$ and hence the message $S$.

Finally, we have that

[TABLE]

[TABLE]

5 MRD codes and coset coding schemes with optimal communication overheads

In this section, we apply Theorem 1 to pairs of Gabidulin codes [8, 21] and their cartesian products [7]. The first family yields optimal coset coding schemes when $n\leq m$ in the sense of (4) and (5), and the second family constitutes a family of maximum rank distance (MRD) codes when $n>m$ [7, Cor. 1].

We recall the definition of MRD codes for convenience of the reader. The Singleton bound for an arbitrary (linear or not) code $\mathcal{C}\subseteq\mathbb{F}_{q^{m}}^{n}$ was first given in [4, Th. 6.3]:

[TABLE]

We then say that $\mathcal{C}$ is MRD if equality holds in (10). In another direction, a Singleton bound on the relative minimum rank distance of a pair of $\mathbb{F}_{q^{m}}$-linear codes $\mathcal{C}_{2}\subsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$ was first given in [16, Prop. 3]:

[TABLE]

Thus if $\mathcal{C}_{1}$ is MRD and $n\leq m$, then equality is satisfied in (11).

5.1 Coset coding schemes based on Gabidulin codes

In this subsection we will make use of Gabidulin codes, which were introduced independently in [8, Sec. 4] and [21, Sec. III]. Throughout this subsection, we will assume that $n\leq m$.

Definition 9** ([8, 21]).**

Fix a basis $\gamma_{1},\gamma_{2},\ldots,\gamma_{m}$ of $\mathbb{F}_{q^{m}}$ as a vector space over $\mathbb{F}_{q}$, and let $0\leq k\leq n$. The Gabidulin code of dimension $k$ and length $n$ over $\mathbb{F}_{q^{m}}$, constructed from the previous basis, is the $\mathbb{F}_{q^{m}}$-linear code $\mathcal{G}_{k}\subseteq\mathbb{F}_{q^{m}}^{n}$ with parity-check matrix given by

[TABLE]

It was proven in [8, Th. 6] and [21, Th. 2] that the code $\mathcal{G}_{k}\subseteq\mathbb{F}_{q^{m}}^{n}$ satisfies

[TABLE]

constituting thus a family of MRD codes covering all parameters when $n\leq m$. Moreover it is clear from the definition that, for a fixed basis of $\mathbb{F}_{q^{m}}$ over $\mathbb{F}_{q}$, they form a nested sequence of codes:

[TABLE]

Thus the next theorem follows directly from Theorem 1:

Theorem 2**.**

Choose integers $k_{2},k_{1},t_{0}$ and $\rho_{0}$ such that $0\leq k_{2}<k_{1}\leq n$ and $2t_{0}+\rho_{0}=n-k_{1}$, and choose any subset $\mathcal{D}\subseteq[n-\rho_{0},n]$ such that $n-\rho_{0}\in\mathcal{D}$.

Now, fix a basis of $\mathbb{F}_{q^{m}}$ over $\mathbb{F}_{q}$, let $\mathcal{C}_{2}\subsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$ be $\mathbb{F}_{q^{m}}$-linear Gabidulin codes of dimensions $k_{2}$ and $k_{1}$ (that is, $\mathcal{G}_{k_{2}}$ and $\mathcal{G}_{k_{1}}$), respectively, and denote the elements in $\mathcal{D}$ by $d_{h}=n-\rho_{0}<d_{h-1}<\ldots<d_{2}<d_{1}$.

The coset coding scheme $F:\mathbb{F}_{q^{m}}^{\alpha\times\ell}\longrightarrow\mathbb{F}_{q^{m}}^{\alpha\times n}$ in Theorem 1 based on this pair of codes and the subsequence of (13) given by the Gabidulin codes $\mathcal{C}^{(j)}=\mathcal{G}_{d_{j}-2t_{0}}$, that is, $k^{(j)}=d_{j}-2t_{0}$, for $j=1,2,\ldots,h$, satisfies $\ell=k_{1}-k_{2}$, is universally $t$-error and $\rho$-erasure-correcting if $2t+\rho\leq n-k_{1}$, and is universally secure under $\mu$ observations if $\mu\leq k_{2}$. In particular, the scheme is optimal in the sense of (4). Moreover, it holds that

[TABLE]

In addition, for any $d\in\mathcal{D}$ and any full-rank matrix $A\in\mathbb{F}_{q}^{d\times n}$, there exist preprocessing functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\ell\alpha m/(d-2t_{0}-k_{2})}$, for $i=1,2,\ldots,d$, which are $t_{0}$-error-correcting and satisfying equality in (5), hence having optimal communication overheads for all $d\in\mathcal{D}$.

Observe that the packet length $m$ of the original Gabidulin codes is multiplied by $\alpha$, which depends only on the maximum number of observations, the number of correctable errors and the set of possible erasures $\mathcal{D}$.

However, there are instances as Example 1 where, due to a particular subpacketization, we need not expand the packet length, hence we obtain a strict improvement on the communication overheads at no cost on the rest of the parameters.

We now give the details of Example 1 and Example 2, which share the same construction: With the given parameters, the construction in [22, Th. 11] gives $\ell=16$ by choosing $k_{1}=24$ and $k_{2}=8$. However, decomposing the packet length as $\alpha m=2048$, with $m=64$ and $\alpha=32$, we may choose $\mathcal{D}=\{24,40\}$, $k^{(1)}=40$, $k_{1}=24$ and $k_{2}=8$, thus $\alpha_{1}=32$, $\alpha_{2}=16$, and $\alpha=32$, and the example follows.

5.2 Coset coding schemes based on MRD cartesian products

In this subsection, we will make use of cartesian products of Gabidulin codes, which yield again MRD codes, but in the case $n>m$, in contrast with plain Gabidulin codes as in the previous subsection. To the best of our knowledge, this is the only known family of MRD $\mathbb{F}_{q^{m}}$-linear codes in $\mathbb{F}_{q^{m}}^{n}$ when $n>m$.

Throughout this subsection, we will assume that $n=lm$, for some positive integer $l$. Take another integer $1\leq k\leq m$, and consider the cartesian product

[TABLE]

where $\mathcal{G}_{k}\subseteq\mathbb{F}_{q^{m}}^{m}$ is a Gabidulin code as in Definition 9. It is proven in [7, Cor. 1] that

[TABLE]

and therefore $\mathcal{C}$ is MRD. Since the codes $\mathcal{G}_{k}$ can be taken in a nested sequence for a fixed basis of $\mathbb{F}_{q^{m}}$ over $\mathbb{F}_{q}$, as in Equation (13), the next result also follows directly from Theorem 1:

Theorem 3**.**

Choose integers $k_{1},k_{2},t_{0}$ and $\rho_{0}$ such that $0\leq k_{2}<k_{1}\leq m$ and $2t_{0}+\rho_{0}=m-k_{1}$, and choose any subset $\mathcal{D}\subseteq[n-\rho_{0},n]$ with elements $d_{h}=n-\rho_{0}<d_{h-1}<\ldots<d_{2}<d_{1}$.

Define $k^{(j)}=d_{j}-(l-1)m-2t_{0}$ and the $\mathbb{F}_{q^{m}}$-linear codes

[TABLE]

for $j=1,2,\ldots,h$, and observe that $k^{(h)}=k_{1}$, hence $\mathcal{C}^{(h)}=\mathcal{C}_{1}=\mathcal{G}_{k_{1}}^{l}$.

The coset coding scheme $F:\mathbb{F}_{q^{m}}^{\alpha\times\ell}\longrightarrow\mathbb{F}_{q^{m}}^{\alpha\times n}$ in Theorem 1 based on these codes satisfies $\ell=l(k_{1}-k_{2})$, is universally $t$-error and $\rho$-erasure-correcting if $2t+\rho\leq m-k_{1}$, and is universally secure under $\mu$ observations if $\mu\leq k_{2}$. Moreover, it holds that

[TABLE]

In addition, for any $d\in\mathcal{D}$ and any full-rank matrix $A\in\mathbb{F}_{q}^{d\times n}$, there exist preprocessing functions $E_{A,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\ell\alpha m/(l(d-2t_{0}-k_{2})-(l-1)n)}$, for $i=1,2,\ldots,d$, which are $t_{0}$-error-correcting and such that

[TABLE]

Observe that the particular case $l=1$ corresponds to the particular case $n=m$ in Theorem 2.

6 Applications

6.1 Universal secure linear network coding

Consider a network with $n$ outgoing links from a source and $N$ ingoing links to a sink, and where the source wants to transmit $\ell$ packets, encoded into $n$ packets (all of the same length), to the sink. Linear network coding, introduced in [1, 15, 17], consists in sending linear combinations over $\mathbb{F}_{q}$ of the received packets at each node of the network, which increases throughput with respect to storing and forwarding.

In this scenario, link errors and erasures expand through the network and an eavesdropper may obtain linear combinations of the sent packets. Thus if the coefficients of the final linear combinations are known to the receiver, then a linearly coded network, with link errors, erasures and observations, can be modelled as a coherent linearized noisy wire-tap channel [22], as in Definition 1.

Assume that the packet length is at least $n$, and fix positive integers $t$, $\rho$ and $\mu$ with $2t+\rho+\mu<n$ and $\rho\leq N$. In [22, Th. 11] a construction (pairs of Gabidulin codes) is given such that $\ell=n-2t-\rho-\mu$, which is optimal due to (4).

However, assuming that $q$ is big enough and the erasure matrix $A\in\mathbb{F}_{q}^{N\times n}$ (see Definition 1) is taken at random as in [11], then it will be full-rank with high probability and $\rho$ can be thought of as a number of erased ingoing links to the sink, due to noise, link failure or the action of the adversary.

Theorem 2 gives an alternative construction to [22, Th. 11] with optimal $\ell=n-2t-\rho-\mu$, where if more than $n-\rho$ ingoing links to the sink are available, the sink can contact the corresponding nodes after exchanging feedback on the number of available nodes, and reduce the communication overhead (hence the amount of packets received by the sink) to its optimal value in view of (5).

6.2 Secure distributed storage with crisscross errors and erasures

Errors and erasures occurring along several rows and/or columns of a matrix over $\mathbb{F}_{q}$ are called crisscross errors and erasures in the literature, and can happen in memory chips and magnetic tapes, for instance (see [21]). Recently, crisscross error and erasure-correction has gained attention in the context of distributed storage where data is stored in several data centers (columns), which in turn store several blocks of data (rows), where mixed and/or correlated failures may occur (see [14]).

In this work, we consider a storage system where data is stored as an $\alpha m\times n$ matrix over $\mathbb{F}_{q}$, where columns are thought of as data centers that are contacted to obtain information from, and rows are blocks of data expanding across the different data centers and sharing correlated errors. More formally, we consider column erasures (equivalently, data centers being available and contacted) together with crisscross errors and where an eavesdropper may listen to a number of columns (data centers).

We formalize crisscross error-correction in the following definitions, which we take from [21, Sec. I]:

Definition 10** (Crisscross weights [21]).**

A cover of a matrix $E\in\mathbb{F}_{q}^{\alpha m\times n}$ is a pair of sets $X\subseteq[\alpha m]$ and $Y\subseteq[n]$ such that if $e_{i,j}\neq 0$, then $i\in X$ or $j\in Y$. We then define the crisscross weight of $E$ as

[TABLE]

We may then formalize crisscross error and erasure-correction, together with security, as follows:

Definition 11**.**

For a subset $I\subseteq[n]$, define the matrix $P_{I}\in\mathbb{F}_{q}^{\#I\times n}$ as that constituted by the rows of the $n\times n$ identity matrix indexed by $I$. We say that the coset coding scheme $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$ is:

Crisscross $t$-error and $\rho$-erasure-correcting if, for every $I\subseteq[n]$ with $\#I=n-\rho$, there exist a decoding function $D_{I}:\mathbb{F}_{q}^{\alpha m\times(n-\rho)}\longrightarrow\mathbb{F}_{q}^{\alpha m\times\ell}$ such that

[TABLE]

for all $C\in\mathcal{C}_{S}$, all $E\in\mathbb{F}_{q}^{\alpha m\times(n-\rho)}$ with ${\rm wt}_{c}(E)\leq t$, and all $S\in\mathbb{F}_{q}^{\alpha m\times\ell}$. 2. 2.

Secure under $\mu$ column-observations if

[TABLE]

for any matrix $W\in\mathbb{F}_{q}^{\alpha m\times\mu}$ constituted by $\mu$ columns of $C=F(S)$, for all $S\in\mathbb{F}_{q}^{\alpha m\times\ell}$.

In this scenario, pieces of data correspond to columns, instead of linear combinations of columns, hence we will consider preprocessing functions $E_{I,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$ depending on a subset of columns $I\subseteq[n]$, where $i\in I$. Hence we may formalize the crisscross error-correction capability of preprocessing functions as follows:

Definition 12**.**

For a subset $I\subseteq[n]$ with $d=\#I$, the preprocessing functions $E_{I,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d$, are $t$-crisscross error-correcting with respect to $F$ if there exists a decoding function $D_{I}:\prod_{i=1}^{d}\mathbb{F}_{q}^{\beta_{i}m}\longrightarrow\mathbb{F}_{q}^{\alpha m\times\ell}$ such that

[TABLE]

where $C\in\mathcal{C}_{S}$ and $\mathbf{c}_{i}$ denotes the $i$-th column of $C$, for $i=1,2,\ldots,d$, for all $S\in\mathbb{F}_{q}^{\alpha m\times\ell}$ and all error matrices $E\in\mathbb{F}_{q}^{\alpha m\times d}$ of crisscross weight at most $t$ with columns $\mathbf{e}_{1},\mathbf{e}_{2},\ldots,\mathbf{e}_{d}\in\mathbb{F}_{q}^{\alpha m}$.

The decoding bandwidth and communication overhead of such functions are defined as in Definition 8.

We now see that the bounds (4) and (5) also hold in this context:

Proposition 3**.**

If the coset coding scheme $F:\mathbb{F}_{q}^{\alpha m\times\ell}\longrightarrow\mathbb{F}_{q}^{\alpha m\times n}$ is crisscross $t$-error and $\rho$-erasure-correcting, and secure under $\mu$ column-observations, then

[TABLE]

Moreover, for a subset $I\subseteq[n]$ with $d=\#I$ and preprocessing functions $E_{I,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\beta_{i}m}$, for $i=1,2,\ldots,d$, that are $t$-crisscross error-correcting with respect to $F$, it holds that:

[TABLE]

Proof.

Define $\rho^{\prime}=\rho+2t$. We will prove that $F$ is $\rho^{\prime}$-crisscross erasure-correcting. If it is not, then there exists a subset $I\subseteq[n]$ with $\#I=n-\rho^{\prime}$, and there exist $C_{1}\in\mathcal{C}_{S_{1}}$ and $C_{2}\in\mathcal{C}_{S_{2}}$, where $S_{1}\neq S_{2}$, such that

[TABLE]

Next take a set of the form $I_{1}=I\cup J_{1}\cup J_{2}$, where $\#I_{1}=n-\rho$ and $t=\#J_{1}=\#J_{2}$ (recall that $n-\rho=n-\rho^{\prime}+2t$). There exist matrices $E_{1},E_{2}\in\mathbb{F}_{q}^{\alpha m\times(n-\rho)}$ of crisscross weight at most $t$ such that

[TABLE]

Hence $F$ cannot be crisscross $t$-error and $\rho$-erasure-correcting, and we reach a contradiction. Now, this implies that $F$ is a classical secret sharing scheme with alphabet $\mathbb{F}_{q}^{\alpha m}$, reconstruction $\rho^{\prime}$ and privacy $\mu$. Thus it follows directly from [13, Th. 1] that

[TABLE]

and we are done.

Finally, the bound (18) can be proven in the same way as the bound (5). ∎

To conclude, we observe that a coset coding scheme, together with preprocessing functions, which are universally (rank) error and erasure-correcting and universally secure in the sense of Definitions 3 and 7 are also crisscross erasure and error-correcting and secure under a given number of column observations in the sense of Definitions 11 and 12, with exactly the same parameters. Thus all constructions in this paper can be directly translated into the context of this subsection.

For illustration purposes, we show how to translate Theorem 2 to this context, thus obtaining coset coding schemes which are optimal in the sense of (17) and (18) for all parameters, whenever $n\leq m$.

Corollary 1**.**

Assume $n\leq m$, choose integers $k_{2},k_{1},t_{0}$ and $\rho_{0}$ such that $0\leq k_{2}<k_{1}\leq n$ and $2t_{0}+\rho_{0}=n-k_{1}$, and choose any subset $\mathcal{D}\subseteq[n-\rho_{0},n]$ with elements $d_{h}=n-\rho_{0}<d_{h-1}<\ldots<d_{2}<d_{1}$.

The coset coding scheme $F:\mathbb{F}_{q^{m}}^{\alpha\times\ell}\longrightarrow\mathbb{F}_{q^{m}}^{\alpha\times n}$ in Theorem 2 with these parameters satisfies $\ell=k_{1}-k_{2}$, is crisscross $t$-error and $\rho$-erasure-correcting if $2t+\rho\leq n-k_{1}$, and is secure under $\mu$ column-observations if $\mu\leq k_{2}$. In particular, the scheme is optimal in the sense of (17). Moreover, it holds that

[TABLE]

In addition, for any $d\in\mathcal{D}$ and any subset $I\subseteq[n]$ with $d=\#I$, there exist preprocessing functions $E_{I,i}:\mathbb{F}_{q}^{\alpha m}\longrightarrow\mathbb{F}_{q}^{\ell\alpha m/(d-2t_{0}-k_{2})}$, for $i=1,2,\ldots,d$, which are $t_{0}$-crisscross error-correcting and satisfying equality in (18), hence having optimal communication overheads for all $d\in\mathcal{D}$.

Observe that optimal crisscross error and erasure-correcting coding schemes can also be obtained by using maximum distance separable (MDS) codes in $\mathbb{F}_{q}^{\alpha m\times n}$, by identifying this vector space with $\mathbb{F}_{q}^{\alpha mn}$, as noticed in [21]. However, such constructions may require extremely large finite fields, for instance $q>\alpha mn$ for Reed-Solomon codes, whereas rank-metric codes allow to obtain optimal coding schemes with the only constraint $n\leq m$, being $q$ unrestricted, allowing in particular using binary fields ($q=2$).

7 Conclusion and open problems

In this paper, we have studied the problem of reducing the communication overhead on a noisy wire-tap channel or storage system where data is encoded as a matrix. The method developed in Section 4 allows to reduce the communication overhead, when more columns are available, at the cost of expanding the packet length (number of rows). However, in the optimal case of pairs of Gabidulin codes (Section 5), strict improvements on the communication overheads are possible at no cost on the rest of the parameters, as shown in Example 1 for practical instances in the applications. We leave as open problem to study when the packet length need not be expanded. Another interesting open problem is to extend our method to codes that are linear over the base field $\mathbb{F}_{q}$, instead of the extension field $\mathbb{F}_{q^{m}}$. This would allow to use all possible MRD codes [4].

Appendix A Proof of Lemma 3

Fix $\mathbb{F}_{q^{m}}$-linear codes $\mathcal{C}_{2}\varsubsetneqq\mathcal{C}_{1}\subseteq\mathbb{F}_{q^{m}}^{n}$ and a matrix $B\in\mathbb{F}_{q}^{\mu\times n}$ in the rest of the appendix.

We start with an auxiliary result, which is a particular case of [18, Th. 3]:

Lemma 4** ([18]).**

It holds that

[TABLE]

where ${\rm Row}(A)\subseteq\mathbb{F}_{q^{m}}^{n}$ denotes the $\mathbb{F}_{q^{m}}$-linear vector space generated by the rows of the matrix $A\in\mathbb{F}_{q}^{\nu\times n}$.

Given an $\mathbb{F}_{q^{m}}$-linear code $\mathcal{C}\subseteq\mathbb{F}_{q^{m}}^{n}$, consider the map $\mathcal{C}\longrightarrow\mathcal{C}B^{T}$ defined by $\mathbf{c}\mapsto\mathbf{c}B^{T}$, for $\mathbf{c}\in\mathcal{C}$. It is surjective and its kernel is $\mathcal{C}\cap\left(\mathcal{V}^{\perp}\right)$, where $\mathcal{V}={\rm Row}(B)$. Therefore

[TABLE]

Using this equation and computing dimensions, it follows that

[TABLE]

Now, using that ${\rm Rk}(B)<d_{R}\left(\mathcal{C}_{2}^{\perp},\mathcal{C}_{1}^{\perp}\right)$ and the previous lemma, it holds that $\mathcal{C}_{2}^{\perp}\cap\mathcal{V}=\mathcal{C}_{1}^{\perp}\cap\mathcal{V}$. Hence the result follows by (19).

Acknowledgement

The author gratefully acknowledges the support from The Danish Council for Independent Research (Grant No. DFF-4002-00367 and Grant No. DFF-5137-00076B “EliteForsk-Rejsestipendium”), and is thankful for the guidance of his advisors Olav Geil and Diego Ruano. This manuscript was written in part when the author was visiting the University of Toronto. He greatly appreciates the support and hospitality of Frank R. Kschischang, and is thankful for valuable discussions on this work.