Efficient and error-tolerant schemes for non-adaptive complex group testing and its application in complex disease genetics

TL;DR

This paper introduces efficient, error-tolerant algorithms for complex group testing, significantly improving identification time and test efficiency, with applications in genetics and biology.

Contribution

The paper proposes novel schemes for classical and generalized complex group testing that are faster and more error-resilient than existing methods.

Findings

Efficient identification of positive complexes in time proportional to $t imes ext{poly}(d, \\ln n)$.

Reduced number of tests compared to previous approaches in certain cases.

Applicability to complex disease genetics and molecular biology.

Abstract

The goal of combinatorial group testing is to efficiently identify up to $d$ defective items in a large population of $n$ items, where $d \ll n$. Defective items satisfy certain properties while the remaining items in the population do not. To efficiently identify defective items, a subset of items is pooled and then tested. In this work, we consider complex group testing (CmplxGT) in which a set of defective items consists of subsets of positive items (called \textit{positive complexes}). CmplxGT is classified into two categories: classical CmplxGT (CCmplxGT) and generalized CmplxGT (GCmplxGT). In CCmplxGT, the outcome of a test on a subset of items is positive if the subset contains at least one positive complex, and negative otherwise. In GCmplxGT, the outcome of a test on a subset of items is positive if the subset has a certain number of items of some positive complex, and negative otherwise. For CCmplxGT, we present a scheme that efficiently identifies all positive complexes in time $t \times \mathrm{poly}(d, \ln{n})$ in the presence of erroneous outcomes, where $t$ is a predefined parameter. As $d \ll n$, this is significantly better than the currently best time of $\mathrm{poly}(t) \times O(n \ln{n})$. Moreover, in specific cases, the number of tests in our proposed scheme is smaller than previous work. For GCmplxGT, we present a scheme that efficiently identifies all positive complexes. These schemes are directly applicable in various areas such as complex disease genetics, molecular biology, and learning a hidden graph.

Tables

Table 1. Table 1: Comparison with existing work. To simplify notation, we set p = s + r ; 𝖠 0 = O ​ ( d 3.57 ​ ln 6.26 ⁡ n 𝖶 6.26 ​ ( d ​ ln ⁡ n ) + d 6 ​ ln 4 ⁡ n 𝖶 4 ​ ( d ​ ln ⁡ n ) + d 2 ​ ln 3 ⁡ n 𝖶 3 ​ ( d ​ ln ⁡ n ) ) formulae-sequence 𝑝 𝑠 𝑟 subscript 𝖠 0 𝑂 superscript 𝑑 3.57 superscript 6.26 𝑛 superscript 𝖶 6.26 𝑑 𝑛 superscript 𝑑 6 superscript 4 𝑛 superscript 𝖶 4 𝑑 𝑛 superscript 𝑑 2 superscript 3 𝑛 superscript 𝖶 3 𝑑 𝑛 p=s+r;\ \mathsf{A}_{0}=O\left(\frac{d^{3.57}\ln^{6.26}{n}}{\mathsf{W}^{6.26}(d\ln{n})}+\frac{d^{6}\ln^{4}{n}}{\mathsf{W}^{4}(d\ln{n})}+\frac{d^{2}\ln^{3}{n}}{\mathsf{W}^{3}(d\ln{n})}\right) , and α = Ω ​ ( 1 ( r + 1 ) ​ r r ​ ( r + s − 1 r ) ) 𝛼 Ω 1 𝑟 1 superscript 𝑟 𝑟 binomial 𝑟 𝑠 1 𝑟 \alpha=\Omega\left(\frac{1}{(r+1)r^{r}\binom{r+s-1}{r}}\right) .

	Scheme	Conditions placed on $r$ and $s$	Error tolerance	Number of tests $t$	Decoding complexity
CCmplxGT	Abasi [25]	$r\le O\left(\frac{{\ln }^2\left(r+s\right)}{\ln \ln \left(r+s\right)}\right)$	$\le \alpha t$	$O\left(r^{11}{\left(4s\right)}^{r+7}\ln n\right)$	$\mathrm{𝗉𝗈𝗅𝗒}\left(t\right)\cdot O\left(n\ln n\right)$
	Chen et al. [4]	None	$\lfloor \frac{z-1}{2}\rfloor$	$O\left(z{\left(\frac{p}{r}\right)}^r{\left(\frac{p}{s}\right)}^{s}p\ln \frac{n}{p}\right)$	Not available
	Corollary 1	None	$\lfloor \frac{z-1}{2}\rfloor$	$O\left(h_0\times \frac{d^2{\ln }^{2}n}{{𝖶}^2\left(d\ln n\right)}\right)$	$h_0\times {𝖠}_0$
	Corollary 2	None	$\lfloor \frac{z-1}{2}\rfloor$	$O\left(h_0\times \frac{d^2{\ln }^{3}n}{{𝖶}^2\left(d\ln n\right)}\right)$	$h_0\times O\left(\frac{d^2{\ln }^{4}n}{{𝖶}^3\left(d\ln n\right)}\right)$
GCmplxGT	Corollary 3	None	$\lfloor \frac{z-1}{2}\rfloor$	$O\left(h_1\times \frac{d^2{\ln }^{2}n}{{𝖶}^2\left(d\ln n\right)}\right)$	$h_1\times {𝖠}_0$ $+O\left(s{u}^{3}d{q}^3\right)$
	Corollary 4	None	$\lfloor \frac{z-1}{2}\rfloor$	$O\left(h_1\times \frac{d^2{\ln }^{3}n}{{𝖶}^2\left(d\ln n\right)}\right)$	$h_1\times O\left(\frac{d^2{\ln }^{4}n}{{𝖶}^3\left(d\ln n\right)}\right)$ $+O\left(s{u}^{3}d{q}^3\right)$

Equations

$$\mathbf{y}=\mathcal{T}\bullet \mathbf{x}$$

$$={\left[\begin{array}{ccc}{y}_1& \dots & y_t\end{array}\right]}^T,$$

$$\mathbf{y}=\mathcal{M}\odot \mathbf{x}=\left[\begin{array}{c}{\mathcal{M}}_{1,\ast }\odot \mathbf{x}\\ ⋮\\ {\mathcal{M}}_{k,\ast }\odot \mathbf{x}\end{array}\right]=\left[\begin{array}{c}{\bigvee }_{j=1}^n{x}_j\wedge m_{1j}\\ ⋮\\ {\bigvee }_{j=1}^n{x}_j\wedge m_{kj}\end{array}\right]=\underset{\begin{array}{c}j=1,x_j=1\end{array}}{\overset{n}{\bigvee }}{\mathcal{M}}_j=\left[\begin{array}{c}{y}_1\\ ⋮\\ y_k\end{array}\right],$$

$$\mathcal{A}=\left[\begin{array}{c}\mathcal{M}\\ \overline{\mathcal{M}}\end{array}\right]$$

$$\mathcal{T}=\left[\begin{array}{c}{\mathcal{G}}_{1,\ast }\\ \mathcal{A}\times \mathsf{diag}({\mathcal{G}}_{1,\ast })\\ ⋮\\ {\mathcal{G}}_{h,\ast }\\ \mathcal{A}\times \mathsf{diag}({\mathcal{G}}_{h,\ast })\end{array}\right]=\left[\begin{array}{c}{\mathcal{G}}_{1,\ast }\\ \mathcal{M}\times \mathsf{diag}({\mathcal{G}}_{1,\ast })\\ \overline{\mathcal{M}}\times \mathsf{diag}({\mathcal{G}}_{1,\ast })\\ ⋮\\ {\mathcal{G}}_{h,\ast }\\ \mathcal{M}\times \mathsf{diag}({\mathcal{G}}_{h,\ast })\\ \overline{\mathcal{M}}\times \mathsf{diag}({\mathcal{G}}_{h,\ast })\end{array}\right]$$

$$\mathbf{y}=\mathcal{T}\bullet \mathbf{x}=\left[\begin{array}{c}\mathsf{test}(\mathsf{supp}({\mathcal{G}}_{1,\ast })\cap \mathsf{supp}(\mathbf{x}))\\ \mathcal{A}\bullet {\mathbf{x}}_1\\ ⋮\\ \mathsf{test}(\mathsf{supp}({\mathcal{G}}_{h,\ast })\cap \mathsf{supp}(\mathbf{x}))\\ \mathcal{A}\bullet {\mathbf{x}}_h\end{array}\right]=\left[\begin{array}{c}\mathsf{test}(\mathsf{supp}({\mathbf{x}}_1))\\ \mathcal{M}\bullet {\mathbf{x}}_1\\ \overline{\mathcal{M}}\bullet {\mathbf{x}}_1\\ ⋮\\ \mathsf{test}(\mathsf{supp}({\mathbf{x}}_h))\\ \mathcal{M}\bullet {\mathbf{x}}_h\\ \overline{\mathcal{M}}\bullet {\mathbf{x}}_h\end{array}\right]=\left[\begin{array}{c}{y}_1\\ {\mathbf{y}}_1\\ {\overline{\mathbf{y}}}_1\\ ⋮\\ y_h\\ {\mathbf{y}}_h\\ {\overline{\mathbf{y}}}_h\end{array}\right]=\left[\begin{array}{c}{y}_1\\ {\mathbf{z}}_1\\ ⋮\\ y_h\\ {\mathbf{z}}_h\end{array}\right]$$

$$h\times (O(k)+\mathsf{A}+d\times O(\mathsf{B}))+O(dh)=O(h(\mathsf{A}+d\mathsf{B})).$$

$$h$$

$$\mathsf{A}$$

$$h$$

$$\mathsf{A}$$

$$q=\sum _{a=1}^s\left(\genfrac{}{}{0ex}{}{|D_a|}{u_a}\right).$$

$$v\times \left(\frac{|C_i|(|C_i|-1)}{2}\times ud\times u\times u|C_i|+|C_i|\right)\le O(s{u}^{3}d{q}^3).$$

$$O(h(\mathsf{A}+d\mathsf{B}))+O(s{u}^{3}d{q}^3)=O(h(\mathsf{A}+d\mathsf{B}))+O\left(s{u}^{3}d{\left(\sum _{a=1}^s\left(\genfrac{}{}{0ex}{}{|D_a|}{u_a}\right)\right)}^3\right).$$

$$h$$

$$\mathsf{A}$$

$$h$$

$$\mathsf{A}$$

Full text

For CCmplxGT, we present a scheme that efficiently identifies all positive complexes in time $t\times\mathsf{poly}(d,\ln{n})$ in the presence of erroneous outcomes, where $t$ is a predefined parameter. As $d\ll n$, this is significantly better than the currently best time of $\mathsf{poly}(t)\times O(n\ln{n})$. Moreover, in specific cases, the number of tests in our proposed scheme is smaller than previous work. For GCmplxGT, we present a scheme that efficiently identifies all positive complexes. These schemes are directly applicable in various areas such as complex disease genetics, molecular biology, and learning a hidden graph.

1 Introduction

The task of combinatorial group testing is to efficiently identify up to $d$ defective items in a large population of $n$ items, where $d$ is usually much smaller than $n$. Defective items are the items that satisfy certain properties while the remaining items in the population, which are referred to as negative items, do not. To efficiently identify the defective items, the items in a set are pooled into several (overlapping) subsets, and then the items in each subset are tested to determine whether they satisfy the properties. The outcome of a test on a subset is either positive, i.e., the subset satisfies the properties, or negative, i.e., the subset does not satisfy the properties. The procedure used to design the tests and obtain the outcomes is called “encoding,” and the procedure used to identify the defective items is called “decoding.”

There are two main approaches to group testing: adaptive and non-adaptive. In adaptive testing, the design of a test depends on the designs of the previous tests. This approach is time-consuming for implementation. Non-adaptive group testing (NAGT) reduces the testing time because all tests are designed in advance and performed in parallel. The focus here is on NAGT.

In classical group testing (CGT) [1], the outcome of a test on a subset of items is positive if the subset has at least one defective item, and negative otherwise. The definition of a test outcome has been generalized to threshold group testing (TGT) with threshold $u$ [2], denoted as $u$-TGT, in which the outcome of a test on a subset is positive if the subset contains at least $u$ defective items, and negative otherwise.

The most sophisticated group testing is complex group testing (CmplxGT) [3, 4, 5]. CmplxGT originated in molecular biology [6] and is also referred to as “cover-free families” [7] and “learning hidden graphs” [8]. In classical CmplxGT (CCmplxGT) [6], an unknown set of subsets of items $D=\{D_{1},\ldots,D_{s}\}$ is designated as a set of defective items in which each $D_{a}$ is called a positive complex, and the remaining items in the set $[n]\setminus\cup_{a=1}^{s}D_{a}$ are designated as negative items, where $[n]=\{1,\ldots,n\}$. Suppose that $|D_{a}|\leq r$ and $\left|\cup_{a=1}^{s}D_{a}\right|\leq d$. The outcome of a test on a subset of items is positive if the subset contains some positive complex $D_{a}$ and negative otherwise. Bui et al. [9] recently introduced generalized CmplxGT (GCmplxGT) as follows. The outcome of a test on a subset of items is positive if the subset contains at least $u_{a}\leq u\leq r$ items from $D_{a}$ for some $a\in[s]=\{1,\ldots,s\}$ and negative otherwise.

Our objective is to minimize the number of tests and to efficiently identify defective items. For CGT, with non-adaptive design, the number of tests is $O(d^{2}\ln{n})$ [10, 11, 12, 13]. Several schemes [14, 15, 16, 17] have been proposed for efficiently identifying defective items by using $O(d^{1+o(1)}\ln{n})$ tests with decoding time $\mathsf{poly}(d,\ln{n})$. For TGT, most work has focused on the number of tests [2, 18, 19, 20] although work on the decoding procedure has recently proliferated [20, 21, 22, 23].

A matrix is an $(s,r;z]$-disjunct matrix if for any $s+r$ columns, there exist at least $z$ rows where each of the $r$ designated columns has 1s and each of the other $s$ columns has 0s. To tackle CCmplxGT, with non-adaptive design, we make use of the fact that the number of rows of an $(s,r;z]$-disjunct matrix is also the number of tests required in CCmplxGT. Chen et al. [4] and Chin et al. [5] gave two upper bounds on the number of tests. Without considering erroneous outcomes, Abasi et al. [24] reported the first algorithm requiring $O(t^{1+o(1)}\ln{n})$ tests to identify positive complexes in time $\mathsf{poly}(t)\cdot O(n\ln{n})$, where $t$ is the number of rows in an $(s+r,r;1]$-disjunct matrix. By considering erroneous outcomes, Abasi [25] needed $t=\mathsf{poly}(s^{r},\ln{n})$ tests to identify all positive complexes in time $\mathsf{poly}(t)\cdot O(n\ln{n})$.

1.1 Contributions

To the best of our knowledge, this is the first work to focus on both the encoding and decoding procedures for GCmplxGT and to show the connection between CmplxGT and complex disease genetics (CDGs). The third contribution is the presentation of efficient encoding and decoding procedures for CCmplxGT with non-adaptive design in the presence of up to $\lfloor\frac{z-1}{2}\rfloor$ erroneous outcomes.

Let $h_{0}$ and $h_{1}$ be the numbers of rows in a $(d-r,r;z]$-disjunct matrix and a $(d-u,u;z]$-disjunct matrix. With CCmplxGT, all positive complexes can be identified using $h_{0}\times O\left(\frac{d^{2}\ln^{4}{n}}{\mathsf{W}^{3}(d\ln{n})}\right)$ tests in time $h_{0}\times\mathsf{poly}(d,\ln{n})$, where $\mathsf{W}(x)=\Theta\left(\ln{x}-\ln{\ln{x}}\right)$. With GCmplxGT, it takes $h_{1}\times O\left(\frac{d^{2}\ln^{4}{n}}{\mathsf{W}^{3}(d\ln{n})}\right)$ tests to identify all positive complexes in time $h_{1}\times\mathsf{poly}(d,\ln{n})+O(su^{3}dq^{3})$, where $q=\sum_{a=1}^{s}\binom{|D_{a}|}{u_{a}}$. Our results are directly applicable in various areas such as complex disease genetics, molecular biology, and learning a hidden graph.

1.2 Comparison

A detailed comparison with previous work is shown in Table 1 ($h_{0}$, $h_{1}$, and $q$ are defined in Section 1.1). Without considering the decoding procedure, Chen et al. [4] showed that the number of non-adaptive tests is $O\left(z\left(\frac{p}{r}\right)^{r}\left(\frac{p}{s}\right)^{s}p\ln{\frac{n}{p}}\right)$, where $p=s+r$ and $\lfloor\frac{z-1}{2}\rfloor$ is the maximum number of erroneous outcomes. Abasi [25] considered a fraction of the errors in test outcomes under the conditions $r<s$ and $r\leq O\left(\frac{\ln^{2}{p}}{\ln{\ln{p}}}\right)$. Abasi showed that all positive complexes can be identified in time $\mathsf{poly}(t)\times O(n\ln{n})$, where $t=O(r^{11}(4s)^{r+7}\ln{n})$. Our proposed scheme has no constraints on either $r$ or $s$. We have shown that all positive complexes can be identified with $h_{0}\times O\left(\frac{d^{2}\ln^{4}{n}}{\mathsf{W}^{3}(d\ln{n})}\right)$ tests in time $h_{0}\times\mathsf{poly}(d,\ln{n})$ with up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes. Our decoding time is thus better than Abasi’s work once $n$ is large enough. Moreover, when $s>d$, the number of tests in our proposed scheme is also smaller than the one proposed by Abasi.

With GCmplxGT, all positive complexes can be identified in time $h_{1}\cdot\mathsf{poly}(d,\ln{n})+O(su^{3}dq^{3})$ by using $h_{1}\cdot O\left(\frac{d^{2}\ln^{4}{n}}{\mathsf{W}^{3}(d\ln{n})}\right)$ tests with up to $\lfloor\frac{z-1}{2}\rfloor$ erroneous outcomes.

1.3 Applications

1.3.1 Complex disease genetics

Complex diseases, e.g., Alzheimer’s disease and Parkinson’s disease, are caused by a combination of genetics and other factors, most of which have not been identified [26]. This work addresses only CDGs [27] that are caused by a combination of many genes. Our goal is to efficiently identify those genes via biological data such as protein-protein interaction (PPI).

Let us denote a gene contributing to a disease as a defective gene. There are several genes contributing to the disease, though the number of non-defective genes outnumbers of the number of defective genes. Let us call a set of these genes a positive complex and a set of genes a complex. A complex may contain a positive complex and there might be more than one positive complex. The outcome of a PPI test on a complex is positive, i.e., the disease occurs, if there are a certain number of defective genes in a positive complex jointly appearing in the complex, and negative otherwise. The problem turns into CmplxGT and is thus resolvable with our proposed scheme.

1.3.2 Molecular biology

Torney [6] introduced a problem in molecular biology as follows. Consider a set $N$ of $n$ molecules. Let $D=\{D_{1},\ldots,D_{s}\}$ be an unknown set of subsets of molecules to be identified. The molecules in each subset $D_{a}$ cause a certain biological phenomenon. A subset of $N$ is called a complex, and each complex $D_{a}$ is called a positive complex. An experiment conducted for any subset of $N$ has two possible outcomes: “positive” if the subset contains at least one $D_{a}$, and “negative” otherwise. Our goal is to identify each positive complex $D_{a}$ such that the number of experiments is as small as possible and the processing time is as short as possible.

The definition of Torney’s problem is identical to the CCmplxGT problem and can be generalized as follows. A certain biological phenomenon occurs if a certain number of molecules in some $D_{a}$ jointly appear. This generalization can be viewed as GCmplxGT and is thus resolvable with our proposed scheme.

1.3.3 Learning a hidden hypergraph

Angluin and Chen [8] described the problem of learning a hidden hypergraph as follows. Consider a set of $n$ items. The objective is to identify an unknown family $D=\{D_{1},\ldots,D_{s}\}$ from the given family $C$ of subsets of $[n]$. The family $C$ is viewed as a hypergraph with the vertex set containing $n$ items. Every $D_{a}$ is considered to be an edge of the hypergraph, and $D$ is a hidden graph. The only operation to be carried out is to test whether a subset of $n$ vertices contains an edge of $D$. Precisely, the outcome of a test on a subset of items is positive if the subset contains all members of some $D_{a}$, and negative otherwise. The goal is to identify the hidden subgraph $D$ in the given hypergraph $C$ with the minimum number of tests. This problem turns into a CCmplxGT problem and is thus resolvable with our proposed scheme.

2 Preliminaries

2.1 Notation

A multiset, denoted with a capital letter with an “$\star$”, is a set that allows multiple instances of its elements. A plain set, denoted with a capital letter with a “$\prime$”, is a set containing indivisible elements only. Function $\mathsf{plain}(\cdot)$ creates a plain set by taking all indivisible elements in the input set. For example, $A^{\star}=\{1,2,2\}$ is a multiset, $D=\{D_{1},D_{2}\}=\{\{1,4\},\{4,8\}\}$ is a set, and $D^{\prime}=\mathsf{plain}(D)=D_{1}\cup D_{2}=\{1,4,8\}$ is a plain set.

For consistency, we use capital calligraphic letters for matrices, non-capital letters for scalars, and bold letters for vectors. All matrices here are binary. The intersection of $l$ columns of a $t\times n$ matrix $\mathcal{T}$ is defined as $\bigwedge_{i=1}^{l}\mathcal{T}_{j_{i}}=\left(\bigwedge_{i=1}^{l}m_{1j_{i}},\ldots,\bigwedge_{i=1}^{l}m_{tj_{i}}\right)^{T}$. Notation $[m]$ represents set $\{1,2,\ldots,m\}$.

Function $\mathsf{diag}(\cdot)$ is used to create a diagonal matrix constructed from the input vector. The support set for $\mathbf{v}=(v_{1},\ldots,v_{w})$ is $\mathsf{supp}(\mathbf{v})=\{j\mid v_{j}\neq 0\}$.

Let $D$ and $D^{\prime}=\mathsf{plain}(D)$ be the defective set consisting of positive complexes and the plain set of $D$. Parameters $n,d$, and $\mathbf{x}=(x_{1},\ldots,x_{n})^{T}$ represent the number of items, the maximum number of defective items, and the representation vector of $n$ items. Finally, let $\mathcal{T}_{i,*},\mathcal{G}_{i,*},\mathcal{M}_{i,*}$, and $\mathcal{M}_{j}$ be row $i$ of matrix $\mathcal{T}$, row $i$ of matrix $\mathcal{G}$, row $i$ of matrix $\mathcal{M}$, and column $j$ of matrix $\mathcal{M}$, respectively.

2.2 Measurement matrix

For vector $\mathbf{x}=(x_{1},\ldots,x_{n})^{T}$, $x_{j}=0$ means that item $j$ is negative, and $x_{j}\neq 0$ means that item $x_{j}$ is defective. Note that $D^{\prime}=\mathsf{supp}(\mathbf{x})$. For a $t\times n$ binary measurement matrix $\mathcal{T}=(t_{ij})$, item $j$ is represented by column $\mathcal{T}_{j}$ and test $i$ is represented by row $\mathcal{T}_{i,*}$; $t_{ij}=1$ if item $j$ belongs to test $i$, and $t_{ij}=0$ otherwise.

Let $\mathsf{test}(S)$ be the test on subset $S\subseteq[n]$. The outcome of the test is either positive (1) or negative (0) and depends on the definition of $D$ and $S$. The non-adaptive tests on $n$ items using $\mathcal{T}$ are defined as

[TABLE]

where $y_{i}=\mathsf{test}\left(\mathsf{supp}(\mathcal{T}_{i,*})\cap\mathsf{supp}(\mathbf{x})\right)$ is the outcome of test $i$ corresponding to row $\mathcal{T}_{i,*}$, and $\bullet$ is the test operator. The procedure to obtain $\mathbf{y}$ is called encoding. The procedure to recover $\mathbf{x}$ from $\mathbf{y}$ and $\mathcal{T}$ is called decoding.

For CGT and TGT, notation $\bullet$ can be explicitly defined and vector $\mathbf{x}$ is viewed as a binary vector in which $x_{j}=1$ (resp., $x_{j}=0$) means item $j$ is defective (resp., negative). With CGT, to avoid ambiguity, we change notation $\bullet$ to $\odot$ and use a $k\times n$ measurement matrix $\mathcal{M}$ instead of the $t\times n$ matrix $\mathcal{T}$. The outcome vector $\mathbf{y}$ in (1) is equal to

[TABLE]

where $y_{i}=\mathcal{M}_{i,*}\odot\mathbf{x}=\bigvee_{j=1}^{n}x_{j}\wedge m_{ij}=\bigvee_{j=1,x_{j}=1}^{n}m_{ij}$ for $i=1,\ldots,k$.

With $u$-TGT, to avoid ambiguity, we change notation $\bullet$ to $\otimes_{u}$. Outcome vector $\mathbf{y}$ in (1) is equal to $\mathbf{y}=\mathcal{T}\otimes_{u}\mathbf{x}=[\mathcal{T}_{1,*}\otimes_{u}\mathbf{x}\ \ldots\ \mathcal{T}_{t,*}\otimes_{u}\mathbf{x}]^{T}=[y_{1}\ \ldots\ y_{t}]^{T}$, where $y_{i}=\mathcal{T}_{i,*}\otimes_{u}\mathbf{x}=1$ if $\sum_{j=1}^{n}x_{j}t_{ij}\geq u$, and $y_{i}=0$ otherwise for $i\in[t]$.

2.3 Disjunct matrices

Disjunct matrices were first introduced by Kautz and Singleton [28] as superimposed codes and then generalized by Stinson and Wei [7] and D’yachkov et al. [29]. The formal definition of a disjunct matrix is as follows.

Definition 1

An $m\times n$ binary matrix $\mathcal{T}$ is called a $(d,r;z]$-disjunct matrix if, for any two disjoint subsets $S_{1},S_{2}\subset[n]$ such that $|S_{1}|=d$ and $|S_{2}|=r$, there exists at least $z$ rows in which there are all 1’s among the columns in $S_{2}$ while all the columns in $S_{1}$ have 0’s, i.e., \left|\bigcap_{j\in S_{2}}\mathsf{supp}\left(\mathcal{T}_{j}\right)\big{\backslash}\bigcup_{j\in S_{1}}\mathsf{supp}\left(\mathcal{T}_{j}\right)\right|\geq z.

Chen et al. [4] gave an upper bound on the number of rows for $(d,u;z]$-disjunct matrices as follows.

Theorem 2.1

[4*, Theorem 3.2]**

For any positive integers $d,u,z$, and $n$ with $p=d+u\leq n$, there exists a $t\times n$ $(d,u;z]$-disjunct matrix with $t=O\left(z\left(\frac{p}{u}\right)^{u}\left(\frac{p}{d}\right)^{d}p\ln{\frac{n}{p}}\right)$.*

When $r=z=1$, a $(d,r;z]$-disjunct matrix becomes a $d$-disjunct matrix. If $\mathcal{M}$ is $d$-disjunct, vector $\mathbf{x}$ can be recovered from $\mathbf{y}=\mathcal{M}\odot\mathbf{x}$. Bui et al. [17] numerically showed that the number of tests in nonrandom construction (each column can be generated without using probability) is optimal for practice (albeit it is not good in term of complexity). Therefore, we use that result here.

Theorem 2.2

[17*, Theorem 8]**

Let $1\leq d\leq n$ be integers. Then there exists a nonrandom $d$-disjunct matrix $\mathcal{M}$ with $k=O\left(\frac{d^{2}\ln^{2}{n}}{\mathsf{W}^{2}(d\ln{n})}\right).$ Each column in $\mathcal{M}$ can be computed in time $O(k^{1.5}/d^{2})$, so matrix $\mathcal{M}$ can be used to identify up to $d^{\prime}$ defective items, where $d^{\prime}\geq\left\lfloor\frac{d}{2}\right\rfloor+1$, in time $O\left(\frac{d^{3.57}\ln^{6.26}{n}}{\mathsf{W}^{6.26}(d\ln{n})}\right)+O\left(\frac{d^{6}\ln^{4}{n}}{\mathsf{W}^{4}(d\ln{n})}\right).$*

The decoding complexity can be reduced by increasing the number of tests:

Theorem 2.3

[17*, Corollary 3]**

Let $1\leq d\leq n$ be integers. There exists a nonrandom $k\times n$ measurement matrix $\mathcal{T}$ with $k=O\left(\frac{d^{2}\ln^{3}{n}}{\mathsf{W}^{2}(d\ln{n})}\right)$, which is used to identify at most $d$ defective items in time $O(k)$. Moreover, each column in $\mathcal{T}$ can be computed in time $O\left(\frac{d\ln^{4}{n}}{\mathsf{W}^{3}(d\ln{n})}\right)$.*

The procedure for obtaining $\mathbf{x}$ from $\mathbf{y}$ is denoted as $\mathsf{supp}(\mathbf{x})=\mathsf{decode}(\mathcal{M},\mathbf{y})$.

3 Problem definitions

We classify CmplxGT into two categories: CCmplxGT and GCmplxGT. Our goal is to identify positive complexes as quickly as possible with a small number of tests. The formal definitions of CCmplxGT and GCmplxGT are given below.

Definition 2 (Classical complex group testing)

Let $1\leq r,s\leq d<n$ be integers. Consider a set $N$ of $n$ items. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $N$, where $|D^{\prime}|=\left|\cup_{a=1}^{s}D_{a}\right|\leq d$, $|D_{a}|\leq r$, and $D_{a}\not\subseteq D_{b}$ for $a\neq b\in[s]$. With CCmplxGT, the outcome of a test on a subset of $N$ is positive if the subset contains some $D_{a}$, and negative otherwise.

We define generalized complex group testing as follows.

Definition 3 (Generalized complex group testing)

Let $1\leq r,s\leq d<n$ be integers. Consider a set $N$ of $n$ items. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $N$, where $|D^{\prime}|=\left|\cup_{a=1}^{s}D_{a}\right|\leq d$, $|D_{a}|\leq r$, and $D_{a}\not\subseteq D_{b}$ for $a\neq b\in[s]$. Any $D_{a}$ is a defective set in $u_{a}$-TGT, where $0<u_{a}\leq u\leq d$ for $a=1,\ldots,s$. With GCmplxGT, the outcome of a test on a subset of $N$ is positive if the subset contains at least $u_{a}$ items in $D_{a}$ for some $a\in[s]$, and negative otherwise.

It is obvious that when $u_{a}=|D_{a}|$ for every $a\in[s]$, Definition 3 reduces to Definition 2. We then define a (incomplete) positive sub-complex as follows.

Definition 4

Consider generalized complex group testing as defined in Definition 3. A set $I$ is called a positive sub-complex if $|I\cap D_{a}|\geq u_{a}$ for some $a\in[s]$. Otherwise, $I$ is called an incomplete positive sub-complex.

For example, let $r=4,s=3,d=7,n=10$ and $N=\{1,2,\ldots,10\}$. Suppose that $D=\{D_{1}=\{1,2\},D_{2}=\{2,3,4\},D_{3}=\{1,3,7,8\}\}$ and $u_{1}=2,u_{2}=2$, and $u_{3}=3$. Then $I_{1}=\{1,2\}$, $I_{2}=\{2,4\}$, and $I_{3}=\{3,7,8\}$ are positive sub-complexes. A few incomplete positive sub-complexes are $I_{4}=\{1,5\},I_{5}=\{3,6,7,8\}$ and $I_{6}=\{1,7,8,9,10\}$.

Because an incomplete positive sub-complex does not affect the outcome of a test on a set containing it, all items in it are considered to be negative for that test. Note that some items in an incomplete defective sub-complex for a test could be defective items in other tests. For example, consider a subset $I_{7}=\{1,2,3,9,10\}$, and with the other settings as the same as in the preceding paragraph. The outcome of a test on $I_{7}$ is positive: items 1 and 2 are identified as defectives, whereas items 3, 9, and 10 are identified as negatives.

4 Review of Bui et al.’s scheme

A part of the scheme proposed by Bui et al. [22] is reviewed here. For $u$-TGT, Bui et al. considered a special case in which the number of defective items equals the threshold; i.e., $|\mathsf{supp}(\mathbf{x})|=u$. Let $\mathcal{M}=(m_{ij})$ be a $k\times n$ $d$-disjunct matrix as described in Section 2.3. Then a measurement matrix is created as

[TABLE]

where $\overline{\mathcal{M}}=(\overline{m}_{ij})$ is the complement matrix of $\mathcal{M}$, and $\overline{m}_{ij}=1-m_{ij}$ for $i=1,\ldots,k$ and $j=1,\ldots,n$. Given measurement matrix $\mathcal{A}$ and a binary representation vector of $u$ defective items $\mathbf{x}$ ($|\mathsf{supp}(\mathbf{x})|=u$), what we observe is $\mathbf{z}=\mathcal{A}\otimes_{u}\mathbf{x}$. The objective is to recover $\mathbf{y}^{\prime}=\mathcal{M}\odot\mathbf{x}=(y^{\prime}_{1},\ldots,y^{\prime}_{k})^{T}$ from $\mathbf{z}$. Then $\mathbf{x}$ can be recovered by using Theorem 2.2 or 2.3.

Assume that the outcome vector is $\mathbf{z}=\mathcal{A}\otimes_{u}\mathbf{x}=\begin{bmatrix}\mathcal{M}\otimes_{u}\mathbf{x}\\ \overline{\mathcal{M}}\otimes_{u}\mathbf{x}\end{bmatrix}=\begin{bmatrix}\mathbf{y}\\ \overline{\mathbf{y}}\end{bmatrix}$, where $\mathbf{y}=\mathcal{M}\otimes_{u}\mathbf{x}=(y_{1},\ldots,y_{k})^{T}$ and $\overline{\mathbf{y}}=\overline{\mathcal{M}}\otimes_{u}\mathbf{x}=(\overline{y}_{1},\ldots,\overline{y}_{k})^{T}$. Then vector $\mathbf{y}^{\prime}=\mathcal{M}\odot\mathbf{x}$ is always obtained from $\mathbf{z}$ by using the following rules: i) If $y_{l}=1$, then $y^{\prime}_{l}=1$; ii) If $y_{l}=0$ and $\overline{y}_{l}=1$, then $y^{\prime}_{l}=0$; and iii) If $y_{l}=0$ and $\overline{y}_{l}=0$, then $y^{\prime}_{l}=1$. Therefore, vector $\mathbf{x}$ can always be recovered. We denote the procedure to get $\mathbf{y}^{\prime}$ by using these three rules as $\mathsf{convert2NACGT}(\mathbf{y})$.

5 Proposed scheme for non-adaptive classical complex group testing

5.1 Encoding procedure

Let $\mathcal{G}$ and $\mathcal{A}$ be an $h\times n$ $(d-r,r;z]$-disjunct matrix and a $2k\times n$ matrix as defined in (3), respectively. On the basis of the final measurement matrix described in [22] and [9], $\mathcal{T}$ is created as follows:

[TABLE]

The vector observed after performing the tests given by $\mathcal{T}$ is

[TABLE]

where $\mathbf{x}_{i}=\mathsf{diag}(\mathcal{G}_{i,*})\times\mathbf{x}$, $y_{i}=\mathsf{test}(\mathsf{supp}(\mathbf{x}_{i}))$, $\mathbf{y}_{i}=\mathcal{M}\bullet\mathbf{x}_{i}=(y_{i1},\ldots,y_{ik})^{T}$, $\overline{\mathbf{y}}_{i}=\overline{\mathcal{M}}\bullet\mathbf{x}_{i}=(\overline{y}_{i1},\ldots,\overline{y}_{ik})^{T}$, and $\mathbf{z}_{i}=[\mathbf{y}_{i}^{T}\ \overline{\mathbf{y}}_{i}^{T}]^{T}$ for $i=1,2,\ldots,h$.

Vector $\mathbf{x}_{i}$ is the vector representing the defective items in row $\mathcal{G}_{i,*}$. Therefore, we have $|\mathsf{supp}(\mathbf{x}_{i})|\leq d$ and $y_{i}=1$ if and only if $D_{a}\subseteq\mathsf{supp}(\mathbf{x}_{i})=\mathsf{supp}(\mathcal{G}_{i,*})\cap\mathsf{supp}(\mathbf{x})$ for some $a\in[s]$. Once $D_{a}\equiv\mathsf{supp}(\mathbf{x}_{i})$, we have $y_{i}=\mathcal{G}_{i,*}\otimes_{|D_{a}|}\mathbf{x}$, $\mathbf{y}_{i}=\mathcal{M}\otimes_{|D_{a}|}\mathbf{x}_{i}$ and $\overline{\mathbf{y}}_{i}=\overline{\mathcal{M}}\otimes_{|D_{a}|}\mathbf{x}_{i}$.

5.2 Decoding procedure

The decoding procedure is summarized as Algorithm 1, where $\mathbf{y}^{\prime}_{i}$ is presumed to be $\mathcal{M}\odot\mathbf{x}_{i}$. The procedure is explained as follows: Step 1 enumerates the $h$ rows of $\mathcal{G}$. Step 2 checks if there is at least one positive complex in row $G_{i,*}$. Steps 3 and 4 calculate $\mathbf{y}^{\prime}_{i}$ and try to recover $\mathbf{x}_{i}$. Step 5 checks if all items from Step 4 form a positive complex. Finally, Step 8 returns all positive complexes.

5.3 Decoding complexity

We summarize Algorithm 1 in the following theorem:

Theorem 5.1

Let $1\leq r\leq d<n$ and $1\leq z,s$ be integers. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $n$ items in CCmplxGT as in Definition 2. Let $\mathcal{G}$ be an $h\times n$ $(d-r,r;z]$-disjunct matrix. Suppose that a $k\times n$ $d$-disjunct matrix $\mathcal{M}$ can be decoded in time $O(\mathsf{A})$ and that each column of $\mathcal{M}$ can be generated in time $O(\mathsf{B})$. A $(2k+1)h\times n$ measurement matrix $\mathcal{T}$, as defined in (4), can thus be used to identify all positive complexes in time $O(h(\mathsf{A}+d\mathsf{B}))$ in the presence of up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes.

5.4 Instantiations of decoding complexity

We instantiate Theorem 5.1 by choosing $\mathcal{G}$ as a $(d-r,r;z]$-disjunct matrix in Theorem 2.1 and $\mathcal{M}$ as a $d$-disjunct matrix in Theorem 2.2.

Corollary 1

Let $1\leq r\leq d<n$ and $1\leq z,s$ be integers. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $n$ items in CCmplxGT as in Definition 2. There exists a $t\times n$ measurement matrix such that $D$ can be identified with $t=O\left(z\left(\frac{d}{r}\right)^{r}\left(\frac{d}{d-r}\right)^{d-r}d\ln{\frac{n}{d}}\cdot\frac{d^{2}\ln^{2}{n}}{\mathsf{W}^{2}(d\ln{n})}\right)$ tests in time $O\left(z\left(\frac{d}{r}\right)^{r}\left(\frac{d}{d-r}\right)^{d-r}d\ln{\frac{n}{d}}\right)\times\mathsf{A}_{0}$ in the presence of up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes, where $\mathsf{A}_{0}$ is defined in Table 1.

To reduce the decoding complexity, matrix $\mathcal{M}$ is chosen as a $d$-disjunct matrix in Theorem 2.3. We then obtain the following corollary, in which the number of tests is larger than that in Corollary 1.

Corollary 2

Let $1\leq r\leq d<n$ and $1\leq z,s$ be integers. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $n$ items in CCmplxGT as in Definition 2. There exists a $t\times n$ measurement matrix such that $D$ can be identified with $t=O\left(z\left(\frac{d}{r}\right)^{r}\left(\frac{d}{d-r}\right)^{d-r}d\ln{\frac{n}{d}}\cdot\frac{d^{2}\ln^{3}{n}}{\mathsf{W}^{2}(d\ln{n})}\right)$ tests in time $O\left(t\times\frac{\ln{n}}{\mathsf{W}(d\ln{n})}\right)$ in the presence of up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes.

6 Proposed scheme for non-adaptive generalized complex group testing

6.1 Encoding procedure

Let $\mathcal{G}$ and $\mathcal{A}$ be an $h\times n$ $(d-u,u;z]$-disjunct matrix and a $2k\times n$ matrix as defined in (3), respectively. Measurement matrix $\mathcal{T}$ and outcome vector $\mathbf{y}$ are created as in Section 5.1.

6.2 Decoding procedure

The decoding procedures is summarized as Algorithm 2. There are two phases in general: one is to identify the defective set $D^{\star}$ (though positive complexes are not identified), and the other one is to identify positive complexes, i.e., identify $D_{a}$ for all $a\in[s]$. The first phase comprises Steps 1 to 12. The second phase comprises Steps 13 to 39. Due to the length limitation, the explanation if the decoding procedure is given in Appendix 0.B.1.

6.3 Decoding complexity

We summarize Algorithm 2 in the following theorem:

Theorem 6.1

Let $1\leq r\leq d<n$ and $1\leq z,s$ be integers. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $n$ items in GCmplxGT as in Definition 3. Let $\mathcal{G}$ be an $h\times n$ $(d-u,u;z]$-disjunct matrix. Suppose that a $k\times n$ $d$-disjunct matrix $\mathcal{M}$ can be decoded in time $O(\mathsf{A})$ and that each column of $\mathcal{M}$ can be generated in time $O(\mathsf{B})$. A $(2k+1)h\times n$ measurement matrix $\mathcal{T}$, as defined in (4), can thus be used to identify all positive complexes in time $O(h(\mathsf{A}+d\mathsf{B})+su^{3}dq^{3})$ in the presence of up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes, where $q=\sum_{a=1}^{s}\binom{|D_{a}|}{u_{a}}$. When $u_{x}\neq u_{y}$ for any $x\neq y\in[s]$, the term $su^{3}dq^{3}$ can be removed.

6.4 Instantiations of decoding complexity

We instantiate Theorem 6.1 by choosing $\mathcal{G}$ as a $(d-u,u;z]$-disjunct matrix in Theorem 2.1 and $\mathcal{M}$ as a $d$-disjunct matrix in Theorem 2.2.

Corollary 3

Let $1\leq r\leq d<n$ and $1\leq z,s$ be integers. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $n$ items in GCmplxGT as in Definition 3. There exists a $t\times n$ measurement matrix such that $D$ can be identified with $t=O\left(z\left(\frac{d}{u}\right)^{u}\left(\frac{d}{d-u}\right)^{d-u}d\ln{\frac{n}{d}}\cdot\frac{d^{2}\ln^{2}{n}}{\mathsf{W}^{2}(d\ln{n})}\right)$ tests in time $O\left(z\left(\frac{d}{u}\right)^{u}\left(\frac{d}{d-u}\right)^{d-u}d\ln{\frac{n}{d}}\right)\times\mathsf{A}_{0}+O(su^{3}dq^{3})$ in the presence of up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes, where $q=\sum_{a=1}^{s}\binom{|D_{a}|}{u_{a}}$ ($\mathsf{A}_{0}$ is defined in Table 1). When $u_{x}\neq u_{y}$ for any $x\neq y\in[s]$, the term $su^{3}dq^{3}$ can be removed.

To reduce decoding complexity, matrix $\mathcal{M}$ is chosen as a $d$-disjunct matrix in Theorem 2.3. We then obtain the following corollary, in which the number of tests is larger than in Corollary 3.

Corollary 4

Let $1\leq r\leq d<n$ and $1\leq z,s$ be integers. Suppose that $D=\{D_{1},\ldots,D_{s}\}$ is an unknown set of positive complexes of $n$ items in GCmplxGT as in Definition 3. There exists a $t\times n$ measurement matrix such that $D$ can be identified with $t=O\left(z\left(\frac{d}{u}\right)^{u}\left(\frac{d}{d-u}\right)^{d-u}d\ln{\frac{n}{d}}\cdot\frac{d^{2}\ln^{3}{n}}{\mathsf{W}^{2}(d\ln{n})}\right)$ tests in time $O\left(t\times\frac{\ln{n}}{\mathsf{W}(d\ln{n})}\right)+O(su^{3}dq^{3})$ in the presence of up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes, where $q=\sum_{a=1}^{s}\binom{|D_{a}|}{u_{a}}$. When $u_{x}\neq u_{y}$ for any $x\neq y\in[s]$, the term $su^{3}dq^{3}$ can be removed.

7 Acknowledgments

The first author thanks Dr. Nikita Polyanskii, Skolkovo Institute of Science and Technology, Russia, for his discussion on hidden hypergraphs.

Appendix 0.A Omitted proofs in Section 5

0.A.1 Correctness of the decoding procedure

Our objective is to recover $\mathbf{x}_{i}$ from $y_{i}$ and $\mathbf{z}_{i}=\begin{bmatrix}\mathbf{y}_{i}\\ \overline{\mathbf{y}}_{i}\end{bmatrix}$ for $i=1,\ldots,h$. We pay attention only to the $\mathbf{x}_{i}$ in which $\mathsf{supp}(\mathbf{x}_{i})\equiv D_{a}$ for some $a\in[s]$ because otherwise $\mathsf{supp}(\mathbf{x}_{i})$ should not be counted as a positive complex.

Step 1 enumerates the $h$ rows of $\mathcal{G}$. We have that $y_{i}$ is the indicator for whether there is at least one positive complex in row $\mathcal{G}_{i,*}$. Because we focus only on rows $\mathcal{G}_{i,*}$ that have exactly one positive complex, vector $\mathbf{z}_{i}$ is not considered if $y_{i}=0$. This is achieved by Step 2.

When $y_{i}=1$, there is at least one positive complex in row $\mathcal{G}_{i,*}$. If there is only one positive complex in row $\mathcal{G}_{i,*}$, say $D_{a}$, then $G_{i}\equiv D_{a}$ for the reason described in the last paragraph of Section 5.1 and the scheme in Section 4. Therefore, the condition in Step 5 does not hold. As a result, the positive complex $G_{i}$ will not be empty after Step 5. Because $\mathcal{G}$ is a $(d-r,r;z]$, each $D_{a}$ is identical to the support set of at least $z$ rows of $\mathcal{G}$ for $a\in[s]$. In other words, there exists at least $z$ rows $\mathcal{G}_{i,*}$s such that $\mathsf{supp}(\mathcal{G}_{i,*})\cap D^{\prime}=D_{a}$ for every $a\in[s]$. Therefore, each $D_{a}$ will be returned more than $\lfloor(z-1)/2\rfloor$ times if there are up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes.

The following argument considers the case in which there are at least two positive complexes present in row $\mathcal{G}_{i,*}$. Our task is now to prevent false defectives, i.e., to prove that Step 5 removes all false defective items. There are two sets of defectives items corresponding to $\mathbf{z}_{i}$: the first one is the true set, which is $S_{i}=\mathsf{supp}(\mathcal{G}_{i,*})\cap D^{\prime}$ and is unknown; the second one is the recovered set $G_{i}$, which is expected to be $S_{i}$ (although not surely). We consider two cases: $|G_{i}\setminus D^{\prime}|=0$ and $|G_{i}\setminus D^{\prime}|>0$, where $D^{\prime}=\mathsf{plain}(D)$. For the latter case, we prove that the condition $\bigwedge_{j\in G_{i}}\mathcal{A}_{j}\not\equiv\mathbf{z}_{i}$ holds. Consider item $j_{0}\in G_{i}\setminus D^{\prime}$. Because $\mathcal{M}$ is a $d$-disjunct matrix, there exists row $\tau$ such that $m_{\tau j_{0}}=1$ and $m_{\tau j}=0$ for all $j\in D^{\prime}$. Hence, $\overline{m}_{\tau j_{0}}=0$ and $\overline{m}_{\tau j}=1$ for all $j\in D^{\prime}$. It follows that $\overline{y}_{i\tau}=1$ because $\overline{m}_{\tau j}=1$ for all $j\in D^{\prime}$ and $D_{a}\subseteq\mathsf{supp}(\overline{\mathcal{M}}_{\tau,*})\cap D^{\prime}$ for some $a\in[s]$. However, $\bigwedge_{j\in G_{i}}\overline{m}_{\tau j}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{0}\}}\overline{m}_{\tau j}\right)\wedge\overline{m}_{\tau j_{0}}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{0}\}}\overline{m}_{\tau j}\right)\wedge 0=0\neq\overline{y}_{i\tau}=1$. Therefore, the condition at Step 5 holds, i.e., the set $G_{i}$ is set to be an empty set and not be counted as a positive complex.

We consider the remaining case when $|G_{i}\setminus D^{\prime}|=0$. In this case, we also prove that the condition at Step 5 holds. Indeed, let $D_{a}$ and $D_{b}$ be two positive complexes in row $\mathcal{G}_{i,*}$. Although there may be more than two positive complexes in row $\mathcal{G}_{i,*}$, we consider only two of them. Let $j_{1}$ be an item in $D_{a}$. Because $\mathcal{M}$ is a $d$-disjunct matrix, there exists row $\chi$ such that $m_{\chi j_{1}}=1$ and $m_{\chi j}=0$ for all $j\in D_{b}$. Hence, $\overline{m}_{\chi j_{1}}=0$ and $\overline{m}_{\chi j}=1$ for all $j\in D_{b}$. Then $\overline{y}_{i\chi}=1$ because $\overline{m}_{\chi j}=1$ for all $j\in D_{b}$ and $D_{b}\subseteq\mathsf{supp}(\overline{\mathcal{M}}_{\chi,*})\cap D^{\prime}$. However, $\bigwedge_{j\in G_{i}}\overline{m}_{\chi j}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{1}\}}\overline{m}_{\chi j}\right)\wedge\overline{m}_{\chi j_{1}}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{1}\}}\overline{m}_{\chi j}\right)\wedge 0=0\neq\overline{y}_{i\chi}=1$. Therefore, if there are at least two positive complexes in row $\mathcal{G}_{i,*}$, the condition at Step 5 holds; i.e., set $G_{i}$ will be set to be an empty set and not be counted as a positive complex.

In conclusion, Algorithm 1 returns all positive complexes with up to $(z-1)/2$ erroneous outcomes.

0.A.2 Decoding complexity

The time to run Step 3 is $O(k)$. Suppose that matrix $\mathcal{M}$ can be decoded in time $O(\mathsf{A})$ and that each column in $\mathcal{M}$ can be generated in time $O(\mathsf{B})$. It is natural that $k\leq O(A),O(B)$ because each column in $\mathcal{M}$ has $k$ entries. It thus takes $O(\mathsf{A})$ time to rune Step 4. Since $\mathcal{M}$ is $d$-disjunct, the cardinality of any $G_{i}$ obtained in Step 5 is not exceeded $d$. Therefore, it takes $d\times O(\mathsf{B})$ time to run Step 5. Step 8 takes $O(d\times h)$ time to run. Because the loop in Step 1 runs $h$ times, the total decoding time is:

[TABLE]

0.A.3 Instantiations of decoding complexity

With the setting in Corollary 1, we have:

[TABLE]

For Corollary 2, we have:

[TABLE]

Appendix 0.B Omitted proofs in Section 6

0.B.1 Decoding procedure

As in Algorithm 1, Step 2 in the first phase enumerates the $h$ rows of $\mathcal{G}$. Step 3 checks if there is at least one positive complex in row $G_{i,*}$. Step 4 calculates $\mathbf{y}^{\prime}_{i}$, which is presumed to be $\mathcal{M}\odot\mathbf{x}_{i}$. Step 5 then tries to recover $\mathbf{x}_{i}$. Steps 6 to 8 accept only a positive sub-complex in which all the elements belong to some positive complex and the cardinality of the sub-complex equals the threshold of the positive complex. Step 11 removes the false positive complexes that may appear after running the previous steps. The final step in the first phase lists all positive sub-complexes without duplicates in Step 12. It follows that $D^{\star}$ becomes a set after this step.

The second phase is to identify positive complexes. As a result of the steps in the first phase, all elements in each subset in $D^{\star}$ belong to some positive complex and the cardinality of the subset equals the threshold of the corresponding positive complex. In the second phase, $D^{\star}$ is first partitioned on the basis of the cardinalities of its subsets, as described in Step 13. Each partition is called a set. Assume there are $v$ sets as in Step 14. When $u_{x}\neq u_{y}$ for any $x\neq y\in[s]$, Steps 15 to 38 can be replaced by Step 37.

Step 15 scans $v$ sets to identify positive complexes. Since two positive complexes could share the same threshold, a set may contain at least two positive complexes. We thus create a flag in Step 16 to indicate whether this happens. If there are at least two positive complexes in a set, our objective is to keep all positive sub-complexes in which the elements belong to a positive complex in the original set and to move the remaining positive sub-complexes into a new set $C_{\mathrm{new}}$. Step 17 declares variable $C_{\mathrm{new}}$ for this case case.

For each set, two subsets, say $A$ and $B$, having a common element fall into two categories: they are two positive sub-complexes of a positive complex or they are two positive sub-complexes of two positive complexes. Steps 18 to 19 scan every pair of subsets in the set. Step 20 checks whether the two subsets intersects and whether they are subsets of any set formed in the previous loops, i.e., $C_{1},\ldots,C_{i-1}$ if we are considering $C_{i}$. A new subset is created by removing an element in $A\cap B$ and adding an element in $A\setminus B$, as described in Steps 21 to 23. Step 23 validates the second category by checking whether the new subset belongs to the set. The indicator for having at least two positive complexes in the set is thus turned on at Step 24. Subset $B$ is then added to the new set $C_{\mathrm{new}}$ in Step 25 and removed from the current set in Step 26. Step 27 adjusts the loop at Step 19 due tos the change in sets in Step 26. If the flag is turned on, Steps 33 to 36 adjust the sets accordingly. Step 37 merges all positive sub-complexes to form a positive complex. Finally, Step 39 returns all positive complexes.

0.B.2 Correctness of decoding procedure

Recall that there are two phases in general: one to identify the defective set $D^{\star}$ (though positive complexes are not identified), and the other to identify positive complexes, i.e., identify $D_{a}$ for $a\in[s]$. The first and second phases consist of Steps 1 to 12 and Steps 13 to 39, respectively. We move to the first phase in details now.

0.B.2.1 Locating defective items

We first assume that there are no erroneous outcomes. Our objective is to recover $\mathbf{x}_{i}$ from $y_{i}$ and $\mathbf{z}_{i}=\begin{bmatrix}\mathbf{y}_{i}\\ \overline{\mathbf{y}}_{i}\end{bmatrix}$ for $i=1,\ldots,h$. We recover only $\mathbf{x}_{i}$ if $|\mathsf{supp}(\mathbf{x}_{i})\setminus D_{a}|=0$ and $|\mathsf{supp}(\mathbf{x}_{i})|=u_{a}$ for some $a\in[s]$. For this condition, we make an ideal assumption that, for every $\mathbf{x}_{i}$, there exists only $D_{a}$ such that $|\mathsf{supp}(\mathbf{x}_{i})\setminus D_{a}|=0$ and $|\mathsf{supp}(\mathbf{x}_{i})|=u_{a}$ for some $a\in[s]$. If this assumption does not hold, $\mathsf{supp}(\mathbf{x}_{i})$ is not added to $D^{\star}$.

Step 1 enumerates the $h$ rows of $\mathcal{G}$. We have that $y_{i}$ is the indicator for whether there is at least one positive sub-complex in row $\mathcal{G}_{i,*}$. If $y_{i}=0$, $\mathsf{supp}(\mathbf{x}_{i})$ is an incomplete positive sub-complex, so $\mathbf{z}_{i}$ is not considered. This is done by running Step 3.

We now consider the case $y_{i}=1$; i.e., there is at least one positive sub-complex in row $\mathcal{G}_{i,*}$. There are three possibilities:

• •

There is only one positive sub-complex in row $\mathcal{G}_{i,*}$ such that $|\mathsf{supp}(\mathbf{x}_{i})\setminus D_{a}|=0$ and $|\mathsf{supp}(\mathbf{x}_{i})|=u_{a}$ for some $a\in[s]$. This is the ideal assumption.

• •

There is only one positive sub-complex in row $\mathcal{G}_{i,*}$ such that $|\mathsf{supp}(\mathbf{x}_{i})\setminus D_{a}|=0$ and $|\mathsf{supp}(\mathbf{x}_{i})|>u_{a}$ for some $a\in[s]$.

• •

There are more than two positive sub-complexes in row $\mathcal{G}_{i,*}$.

There are two sets of defectives items corresponding to $\mathbf{z}_{i}$: the first one is the true set, which is $S_{i}=\mathsf{supp}(\mathbf{x}_{i})$ and is unknown, and the second one is the recovered set $G_{i}$, which is expected to be $S_{i}$ (although not surely). Since we made the ideal assumption, Steps 4 and 5 are simply to implement the procedure described in Section 4. The important point is that $G_{i}\equiv\mathsf{supp}(\mathbf{x}_{i})$ if the ideal assumption holds without knowing the exact value of $u_{a}$ (as long as $u_{a}\leq d$). Step 6 checks whether the ideal assumption holds. If the first possibility occurs, it is obvious that the ideal assumption holds. Step 7 is thus implemented.

We now prove that when the second or third possibility occurs, i.e., the ideal assumption does not hold, the condition in Step 7 does not hold. Consequently, there is no positive sub-complex to be added to the defective set $D^{\star}$.

Consider the second possibility. We break it down into two cases: $|G_{i}\setminus D^{\prime}|>0$ and $|G_{i}\setminus D^{\prime}|=0$, where $D^{\prime}=\mathsf{plain}(D)$. In the first case, the argument is similar to the one in Section 0.A.1. Consider item $j_{0}\in G_{i}\setminus D^{\prime}$. Because $\mathcal{M}$ is a $d$-disjunct matrix, there exists row $\tau$ such that $m_{\tau j_{0}}=1$ and $m_{\tau j}=0$ for all $j\in D^{\prime}$. Hence, $\overline{m}_{\tau j_{0}}=0$ and $\overline{m}_{\tau j}=1$ for all $j\in D^{\prime}$. It follows that $\overline{y}_{i\tau}=1$ because $D_{a}\subseteq\mathsf{supp}(\mathbf{x}_{i})$ for some $a\in[s]$ and $\overline{m}_{\tau j}=1$ for all $j\in D^{\prime}$. However, $\bigwedge_{j\in G_{i}}\overline{m}_{\tau j}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{0}\}}\overline{m}_{\tau j}\right)\wedge\overline{m}_{\tau j_{0}}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{0}\}}\overline{m}_{\tau j}\right)\wedge 0=0\neq\overline{y}_{i\tau}=1$. Therefore, the condition at Step 6 does not hold.

We now consider the remaining case in which $|G_{i}\setminus D^{\prime}|=0$. Using the same argument as above, we consider item $j_{0}\in G_{i}$. Because $\mathcal{M}$ is a $d$-disjunct matrix and $|G_{i}|\leq d$, there exists row $\upsilon$ such that $m_{\upsilon j_{0}}=1$ and $m_{\upsilon j}=0$ for all $j\in G_{i}\setminus\{j_{0}\}$. Hence, $\overline{m}_{\upsilon j_{0}}=0$ and $\overline{m}_{\upsilon j}=1$ for all $j\in D^{\prime}$. Moreover, $D^{\prime}\setminus\{j_{0}\}$ must contain a positive sub-complex because of the condition in the second possibility. Therefore, $\overline{y}_{i\upsilon}=1$. On the other hand, we have $\bigwedge_{j\in G_{i}}\overline{m}_{\upsilon j}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{0}\}}\overline{m}_{\upsilon j}\right)\wedge\overline{m}_{\upsilon j_{0}}=\left(\bigwedge_{j\in G_{i}\setminus\{j_{0}\}}\overline{m}_{\upsilon j}\right)\wedge 0=0\neq\overline{y}_{i\upsilon}=1$. Therefore, the condition at Step 6 does not hold.

For the third possibility, let $D_{a}$ and $D_{b}$ be two positive sub-complexes in row $\mathcal{G}_{i,*}$. Although there might be more than two positive sub-complexes in row $\mathcal{G}_{i,*}$, we only consider two of them. Following the argument in section 0.A.1, there always exists a row $\chi$ such that $\bigwedge_{j\in G_{i}}\overline{m}_{\chi j}=0\neq\overline{y}_{i\chi}=1$. Therefore, the condition at Step 6 does not hold.

After running Steps 1 to 10, every member $M$ of $D^{\star}$ satisfies $|M\setminus D_{a}|=0$ and $|M|=u_{a}$ for some $a\in[s]$. For each $D_{a}$, there are $\binom{|D_{a}|}{u_{a}}$ such $M$ if the frequency of $M$ in $D^{\star}$ is not considered. Because $\mathcal{G}$ is a $(d-u,u;z]$-disjunct matrix and $u_{a}\leq u$, there exists at least $z$ rows $\mathcal{G}_{i,*}$ such that $M=\mathsf{supp}(\mathbf{x}_{i})$.

We now consider erroneous outcomes. Since there are up to $\lfloor(z-1)/2\rfloor$ erroneous outcomes, any false positive sub-complex in $D^{\star}$ cannot appear more than $\lfloor(z-1)/2\rfloor$ times. Therefore, we can eliminate them by checking their frequencies in $D^{\star}$. This sanitization procedure is done by running Step 11. Finally, we keep only one copy of each positive sub-complex in $D^{\star}$ by running Step 12.

In summary, this phase results in a set $D^{\star}$ such that each member $M$ of $D^{\star}$ satisfies $|M\setminus D_{a}|=0$ and $|M|=u_{a}$ for some $a\in[s]$. Moreover, for each $D_{a}$, the number of such $M$ is $\binom{|D_{a}|}{u_{a}}$.

0.B.2.2 Identifying positive complexes

There are two conditions to accomplish this phase:

All elements of every subset of $D^{\star}$ belong to a positive complex such that the cardinality of the subset is equal to the threshold of the positive complex. 2. 2.

For any two subsets of $D^{\star}$, say $A$ and $B$, and $|A|=|B|=c$, if all elements of $A\cup B$ do not belong to a positive complex, there exists an element $x\in A$ and an element $y\in B$ such that $D^{\star}$ does not contain the set $\{x\}\cup B\setminus\{y\}$.

The first condition is accomplished in the first phase. We now prove the second condition. Because of the first condition, there exists $a,b\in[s]$ such that $|A\cap D_{a}|=0$, $|A|=u_{a}=c$, $|B\cap D_{b}|=0$, and $|B|=u_{b}=c$. If all elements of $A\cup B$ do not belong to a positive complex, i.e., $A\cup B\not\subseteq D_{a}$ and $A\cup B\not\subseteq D_{b}$, we must have $a\neq b$ and $|A\cap B|\leq c-1$. Therefore, there exists an element $x\in A$ and an element $y\in B$ such that the set $\{x\}\cup(B\setminus\{y\})$ is an incomplete positive sub-complex. Therefore, $D^{\star}$ does not contain the set $\{x\}\cup(B\setminus\{y\})$.

We now prove that the second phases consisting of Steps 13 to 39 returns all positive complexes. Because of the first condition, all elements of a positive complex are distributed into several subsets in which the cardinality of the subset is equal to the threshold of the positive complex. Step 13 is thus to create sets containing subsets of positive complexes on the basis of their thresholds. After running Step 13, all elements of a positive complex must belong to a set. Therefore, when $u_{x}\neq u_{y}$ for any $x\neq y\in[s]$, Steps 15 to 38 can be replaced by Step 37

Every set created after Step 13 is investigated in Step 15. For a set, says $C$, there are two possibilities: its plain set contains only a positive complex or more than two positive complexes. The second possibility can be detected by using the two conditions. The first condition provides a strategy for forming a positive complex: for a subset in $C$ there always exists another subset such that these two subsets have a common element and their union is a subset of a positive complex. With this strategy, we scan every pair of subsets in a set. This procedure is done by running Steps 18 and 19. If two subsets, say $A$ and $B$, have common elements, they must be subsets of a positive complex or be two subsets of two positive complexes. It is easy to determine into which case these two subsets fall. Step 20 checks whether $A$ and $B$ have common elements and whether they are subsets of any set formed in previous loops, i.e., $C_{1},\ldots,C_{i-1}$ if we are considering set $C_{i}$. Because each subset has at least one element that does not belong to the other, we create a new set $x\cup(A\setminus y)$, where $x\in A\setminus B$ and $y\in A\cap B$. Because the cardinality of the new subset is equal to $|A|$, it must belong to set $C$ if the two subsets $A$ and $B$ are subsets of a positive complex. Otherwise, the new subset does not belong to $C$ because it is an incomplete positive sub-complex. If the two subsets belong to two positive complexes, we simply place one subset into a new set and consider it later. These actions are done in Steps 21 to 29. After these steps are run, set $C$ contains only subsets of a positive complex. Therefore, Step 37 forms that positive complex.

0.B.3 Decoding complexity

Suppose that matrix $\mathcal{M}$ can be decoded in time $O(\mathsf{A})$ and that each column in $\mathcal{M}$ can be generated in time $O(\mathsf{B})$. From the same analysis described in Section 5.2, the complexity of Steps 1 to 11 is $O(h(\mathsf{A}+d\mathsf{B}))$. Because $|\mathsf{plain}(D^{\star})|\leq dh$, Steps 12 to 13 take $O(dh)$ time.

From the first condition in Section 0.B.2.2, the cardinality of $D^{\star}$ is

[TABLE]

Algorithm 2 runs $v\leq s$ loops in Step 15 and takes $|C_{i}|(|C_{i}|-1)/2$ time to run the loops in Steps 18 and 19. Because $u_{a}\leq u$ for all $a\in[s]$, Step 20 takes $O(ud)$ time. Steps 22,  23, and 37 take $O(u)$ loops, $O(u|C_{i}|)$ time, and $O(|C_{i}|)$ time to run, respectively. Because $|D^{\star}|=q$, we have $|C_{i}|\leq q$. Therefore, Steps 15 to 38 take time

[TABLE]

In summary, the total decoding time is

[TABLE]

0.B.4 Instantiations of decoding complexity

With the setting in Corollary 3, we have:

[TABLE]

For Corollary 4, we have:

[TABLE]