Improving Computational Efficiency of Communication for Omniscience and Successive Omniscience

TL;DR

This paper introduces a more efficient algorithm for communication in omniscience problems, reducing complexity from quadratic to linear in the number of users, and extends it to optimize successive omniscience with minimal additional computation.

Contribution

It proposes a parametric algorithm that improves the complexity of minimizing communication rates in omniscience and extends to optimal local omniscience in successive omniscience scenarios.

Findings

Complexity of rate minimization reduced to O(|V| * SFM(|V|)).

The parametric algorithm fully determines the minimum sum-rate solution.

Optimal local omniscience can be achieved with the same complexity using the proposed method.

Abstract

For a group of users in $V$ where everyone observes a component of a discrete multiple random source, the process that users exchange data so as to reach omniscience, the state where everyone recovers the entire source, is called communication for omniscience (CO). We first consider how to improve the existing complexity $O(|V|^2 \cdot \text{SFM}(|V|)$ of minimizing the sum of communication rates in CO, where $\text{SFM}(|V|)$ denotes the complexity of minimizing a submodular function. We reveal some structured property in an existing coordinate saturation algorithm: the resulting rate vector and the corresponding partition of $V$ are segmented in $\alpha$, the estimation of the minimum sum-rate. A parametric (PAR) algorithm is then proposed where, instead of a particular $\alpha$, we search the critical points that fully determine the segmented variables for all $\alpha$ so that they converge to the solution to the minimum sum-rate problem and the overall complexity reduces to $O(|V| \cdot \text{SFM}(|V|))$. For the successive omniscience (SO), we consider how to attain local omniscience in some complimentary user subset so that the overall sum-rate for the global omniscience still remains minimum. While the existing algorithm only determines a complimentary user subset in $O(|V| \cdot \text{SFM}(|V|))$ time, we show that, if a lower bound on the minimum sum-rate is applied to the segmented variables in the PAR algorithm, not only a complimentary subset, but also an optimal rate vector for attaining the local omniscience in it are returned in $O(|V| \cdot \text{SFM}(|V|))$ time.

Equations

$$r(X)=\sum _{i\in X}{r}_i,\forall X\subseteq V$$

$$\mathcal{R}(V)=\{{\mathbf{r}}_V\in {\mathbb{R}}^{|V|}:r(X)\ge H(X|V\setminus X),\forall X⊊V\},$$

$$R_{\text{CO}}(V)=\min \{r(V):{\mathbf{r}}_V\in \mathcal{R}(V)\}.$$

$$R_{\text{CO}}(V)=\underset{\mathcal{P}\in \Pi (V):|\mathcal{P}|>1}{\max }\sum _{C\in \mathcal{P}}\frac{H(V)-H(C)}{|\mathcal{P}|-1},$$

$${\hat{f}}_{\alpha }(V)=\underset{\mathcal{P}\in \Pi (V)}{\min }{f}_{\alpha }[\mathcal{P}].$$

$$\{\{i\}:i\in V\}={\mathcal{P}}^{(p)}\prec \dots \prec {\mathcal{P}}^{(1)}\prec {\mathcal{P}}^{(0)}=\{V\}$$

$$g_{\alpha }(\tilde{\mathcal{X}})=f_{\alpha }(\tilde{\mathcal{X}})-r_{\alpha }(\tilde{\mathcal{X}}),\forall \mathcal{X}\subseteq {\mathcal{Q}}_{\alpha ,V_i},$$

$$\min \{g_{\alpha }(\tilde{\mathcal{X}}):\{i\}\in \mathcal{X}\subseteq {\mathcal{Q}}_{\alpha ,V_i}\}$$

$$=\max \{\xi :{\mathbf{r}}_V+\xi {\chi }_i\in P(f_{\alpha },\le )\}.$$

$${\mathbf{r}}_{\alpha ,V_i}$$

$${\mathcal{Q}}_{\alpha ,V_i}$$

$${\mathsf{Z}}_1$$

$${\mathsf{Z}}_2$$

$${\mathsf{Z}}_3$$

$${\mathsf{Z}}_4$$

$${\mathsf{Z}}_5$$

$${\mathbf{r}}_{\alpha ,V_2}=\{\begin{array}{ll}(\alpha -2,\alpha -4)& \alpha \in [0,4],\\ (\alpha -2,0)& \alpha \in (4,10],\end{array}$$

$${\mathcal{Q}}_{\alpha ,V_2}=\{\begin{array}{ll}\{\{1\},\{2\}\}& \alpha \in [0,4],\\ \{\{1,2\}\}& \alpha \in (4,10],\end{array}$$

$${\tilde{\mathcal{U}}}_{\alpha ,V_2}=\{\begin{array}{ll}\{2\}& \alpha \in [0,4],\\ \{1,2\}& \alpha \in (4,10].\end{array}$$

$${\mathcal{U}}_{\alpha ,V_2}=\{\begin{array}{ll}\{\{2\}\}& \alpha \in [0,4],\\ \{\{1\},\{2\}\}& \alpha \in (4,10].\end{array}$$

$$f_1(Y)-f_1(X)\ge f_2(Y)-f_2(X)$$

$$g_{\alpha }\twoheadrightarrow g_{{\alpha }^{\prime }},\forall \alpha ,{\alpha }^{\prime }:\alpha <{\alpha }^{\prime }.$$

$$g_{\alpha }(\tilde{\mathcal{Y}})-g_{\alpha }(\tilde{\mathcal{X}})-g_{{\alpha }^{\prime }}(\tilde{\mathcal{Y}})+g_{{\alpha }^{\prime }}(\tilde{\mathcal{X}})$$

$$=r_{{\alpha }^{\prime }}(\tilde{\mathcal{Y}}\setminus \tilde{\mathcal{X}})-r_{\alpha }(\tilde{\mathcal{Y}}\setminus \tilde{\mathcal{X}})$$

$$=\{\begin{array}{ll}0& \tilde{\mathcal{X}}=\tilde{\mathcal{Y}},\\ {\hat{f}}_{{\alpha }^{\prime }}(\tilde{\mathcal{Y}}\setminus \tilde{\mathcal{X}})-{\hat{f}}_{\alpha }(\tilde{\mathcal{Y}}\setminus \tilde{\mathcal{X}})& \tilde{\mathcal{X}}⊊\tilde{\mathcal{Y}},\end{array}$$

$$\{i\}={\tilde{\mathcal{S}}}_q⊊\dots ⊊{\tilde{\mathcal{S}}}_1⊊{\tilde{\mathcal{S}}}_0=V_i$$

$${\tilde{\mathcal{U}}}_{\alpha ,V_3}=\{\begin{array}{ll}\{3\}& \alpha \in [0,8],\\ \{1,2,3\}& \alpha \in (8,10]\end{array}$$

$${\mathbf{r}}_{\alpha ,V_3}=\{\begin{array}{ll}(\alpha -2,\alpha -4,\alpha -6)& \alpha \in [0,4],\\ (\alpha -2,0,\alpha -6)& \alpha \in (4,8],\\ (\alpha -2,0,2)& \alpha \in (8,10],\end{array}$$

$${\mathcal{Q}}_{\alpha ,V_3}=\{\begin{array}{ll}\{\{1\},\{2\},\{3\}\}& \alpha \in [0,4],\\ \{\{1,2\},\{3\}\}& \alpha \in (4,8],\\ \{\{1,2,3\}\}& \alpha \in (8,10].\end{array}$$

$${\mathbf{r}}_{\alpha ,V_i}$$

$${\mathcal{Q}}_{\alpha ,V_i}$$

$${\tilde{\mathcal{U}}}_{\alpha ,V_4}=\{\begin{array}{ll}\{4\}& \alpha \in [0,7],\\ \{1,\dots ,4\}& \alpha \in (7,10].\end{array}$$

$${\mathbf{r}}_{\alpha ,V_4}=\{\begin{array}{ll}(\alpha -2,\alpha -4,\alpha -6,\alpha -6)& \alpha \in [0,4],\\ (\alpha -2,0,\alpha -6,\alpha -6)& \alpha \in (4,7],\\ (\alpha -2,0,\alpha -6,8-\alpha )& \alpha \in (7,8],\\ (\alpha -2,0,2,0)& \alpha \in (8,10],\end{array}$$

$${\mathcal{Q}}_{\alpha ,V_4}=\{\begin{array}{ll}\{\{1\},\dots ,\{4\}\}& \alpha \in [0,4],\\ \{\{1,2\},\{3\},\{4\}\}& \alpha \in (4,7],\\ \{\{1,\dots ,4\}\}& \alpha \in (7,10].\end{array}$$

$${\tilde{\mathcal{U}}}_{\alpha ,V}=\{\begin{array}{ll}\{5\}& \alpha \in [0,6],\\ \{1,2,5\}& \alpha \in (6,6.5],\\ \{1,\dots ,5\}& \alpha \in (6.5,10].\end{array}$$

$${\mathbf{r}}_{\alpha ,V}=\{\begin{array}{ll}(\alpha -2,\alpha -4,\alpha -6,\alpha -6,\alpha -5)& \alpha \in [0,4],\\ (\alpha -2,0,\alpha -6,\alpha -6,\alpha -5)& \alpha \in (4,6],\\ (\alpha -2,0,\alpha -6,\alpha -6,1)& \alpha \in (6,6.5],\\ (\alpha -2,0,\alpha -6,\alpha -6,14-2\alpha )& \alpha \in (6.5,7],\\ (\alpha -2,0,\alpha -6,8-\alpha ,0)& \alpha \in (7,8],\\ (\alpha -2,0,2,0,0)& \alpha \in (8,10],\end{array}$$

Full text

For the successive omniscience (SO), we consider how to attain local omniscience in some complimentary user subset so that the overall sum-rate for the global omniscience still remains minimum. While the existing algorithm only determines a complimentary user subset in $O(|V|\cdot\text{SFM}(|V|))$ time, we show that, if a lower bound on the minimum sum-rate is applied to the segmented variables in the PAR algorithm, not only a complimentary subset, but also an optimal rate vector for attaining the local omniscience in it are returned in $O(|V|\cdot\text{SFM}(|V|))$ time.

I introduction

Let there be a finite number of users that are indexed by the set $V$. Each of them observes a distinct component of a discrete multiple random source in private. The users are allowed to exchange their observations over public authenticated noiseless broadcast channels so as to attain omniscience, the state that each user reconstructs all components in the multiple source. This process is called communication for omniscience (CO) [1], where the fundamental problem is how to attain omniscience with the minimum sum of broadcast rates.

The study of the minimum sum-rate problem started in the coded cooperative data exchange (CCDE) [2, 3, 4], where the users obtain finite-length observations of a multiple linear source and the broadcasts are an integral number of linear combinations. The authors in [3, 4] solved the minimum sum-rate problem in CCDE in polynomial time by $O(|V|^{2})$ calls of the submodular function minimization (SFM) algorithm. In the general models where the observation sequences are infinitely long and communication rates are real-valued, it is shown in [5, 6, 7] that the minimum sum-rate is determined by the first critical point in the principal sequence of partitions (PSP) [8, 9], a partition sequence that determines a Dilworth truncation parameterized by an estimation $\alpha$ of the minimum sum-rate. A modified decomposition algorithm (MDA) is proposed in [5, 6] that recursively uses the Dilworth truncation to update the current estimation $\alpha$ until it converges to the minimum. The asymptotic complexity of the MDA is again $O(|V|^{2}\cdot\text{SFM}(|V|))$.111The asymptotic time complexity refers to the asymptotic measure of the (worst-case) time complexity of an algorithm when the number of users $|V|$ grows large [10]. $\text{SFM}(|V|)$ denotes the asymptotic complexity of minimizing a submodular function that is defined on $2^{V}$. However, $\text{SFM}(|V|)$ is in the order of $|V|^{5}$ to $|V|^{8}$ [11, Chapter VI] and yet it is still worth discussing how to reduce this complexity.

In the meantime, the idea of successive omniscience (SO) is proposed in [12, 13] revealing that the CO problem can be solved in a two-stage manner: local omniscience followed by the global omniscience. It is shown in [12] that there is a particular group of complimentary user subsets so that the local omniscience in any of them can be attained first while the overall communication rates for the global omniscience whereafter still remains minimized. Then, the SO boils down to the problem of how to select a complimentary subset and to attain omniscience in it. The authors in [14] derived a sufficient condition for a user subset to be complimentary so that such a subset can be searched in $O(|V|\cdot\text{SFM}(|V|))$ time. However, we are still missing an optimal rate vector for the local omniscience problem. To avoid repetitively applying the existing algorithms for CO,222The local omniscience in a user subset is again a CO problem where the existing algorithms, e.g., the MDA in [5, 6], can be applied. However, in the case of multi-stage SO, we need to call these algorithms time and again. It makes the SO more complex than attaining omniscience in a one-off manner. it is desirable to see if an optimal rate vector for the local omniscience can be obtained at the same time when a complimentary user subset is chosen.

The main purpose of this paper is to improve the computational efficiency for solving both CO and SO problems. We first show how to reduce the complexity of solving the minimum sum-rate problem from $O(|V|^{2}\cdot\text{SFM}(|V|))$ to $O(|V|\cdot\text{SFM}(|V|))$. We start the work by reviewing the coordinate saturation (CoordSat) algorithm,333In this paper, the CoordSat algorithm refers to [6, Algorithm 3]. a nesting algorithm in the MDA in [6] that determines the Dilworth truncation for a given $\alpha$. We show that the partition obtained in each iteration gets coarser in $\alpha$ and its value is segmented by a finite number of critical points that also form a PSP. We then consider a question that follows naturally: how to obtain this partition for all, instead of one, $\alpha$. For the user subset that is used to update the partition,444In the CoordSat algorithm, the partition is updated by merging all elements that intersect with a user subset. we prove a strict strong map property showing that its size is shrinking and also segmented in $\alpha$. All critical values for determining this segmented subset can be searched by the parametric submodular function minimization (PSFM) algorithm [15, 16, 17] that completes at the same asymptotic time as the SFM algorithm. We then propose a parameterized (PAR) algorithm that iteratively updates a segmented partition and a rate vector towards the PSP of the entire user set, which determines the minimum sum-rate, and an optimal rate vector, respectively. The complexity of the PAR algorithm reduces to $O(|V|\cdot\text{SFM}(|V|))$. We also discuss how to implement the PAR algorithm in a distributed manner.

In the second part of this paper, we discuss how to solve the SO problem efficiently. We apply a lower bound on the minimum sum-rate to the segmented partition and rate vector at the end of each iteration of the PAR algorithm to show that not only a complimentary user subset for SO, but also an optimal rate vector that attains the local omniscience in it can both be determined in $O(|V|\cdot\text{SFM}(|V|))$ time.

This paper is organized as follows. The system model for CO is described in Section II, where we also introduce the notation and review the existing results that are required to prove the strong map property and propose the PAR algorithm in Section III. In Section III-D, we show the complexity reduction by the PAR algorithm and discuss a distributed implementation method. In Section IV, we utilize the PAR algorithm to efficiently solve the SO problem.

II Communication for Omniscience

Let $V=\{1,\dotsc,|V|\}$ with $|V|>1$ be a finite set that indexes all users in the system. We call $V$ the ground set. Let $\mathsf{Z}_{V}=(\mathsf{Z}_{i}:i\in V)$ be a vector of discrete random variables indexed by $V$. For each $i\in V$, user $i$ privately observes an $n$-sequence $\mathsf{Z}_{i}^{n}$ of the random source $\mathsf{Z}_{i}$ that is i.i.d. generated according to the joint distribution $P_{\mathsf{Z}_{V}}$. Each user is able to compress and publish his/her observations over noiseless broadcast channels to help the others reconstruct the source sequence $\mathsf{Z}_{V}^{n}$. The state that each user recovers $\mathsf{Z}_{V}^{n}$ is called omniscience and the process that the users communicate with each other to attain omniscience is called communication for omniscience (CO) [1].

II-A Minimum Sum-rate Problem

Let $\mathbf{r}_{V}=(r_{i}:i\in V)$ be a rate vector and $r$ be the sum-rate function associated with $\mathbf{r}_{V}$ such that

[TABLE]

with the convention $r(\emptyset)=0$. We call $\mathbf{r}_{V}$ an achievable rate vector if the omniscience can be attained by letting the users communicate at the rates designated by $\mathbf{r}_{V}$. The authors in [1] derived the achievable rate region in terms of the multiterminal Slepian-Wolf constraint [18, 19]:

[TABLE]

where $H(X)$ is the amount of randomness in $\mathsf{Z}_{X}$ measured by Shannon entropy [20] and $H(X|Y)=H(X\cup Y)-H(Y)$ is the conditional entropy of $\mathsf{Z}_{X}$ given $\mathsf{Z}_{Y}$.

The fundamental problem in CO is to minimize the sum-rate in the achievable rate region [1, Proposition 1]

[TABLE]

To efficiently solve the minimum sum-rate problem by avoiding dealing with the exponentially growing number of constraints in the linear programming, (1) is converted to a combinatorial optimization problem [1, Example 4] [21] [6, Corollary 6]

[TABLE]

where $\Pi(V)$ denotes the set of all partitions of $V$. It is shown in [7, 5, 6] that problem (2) can be solved based on the existing submodular function minimization (SFM) techniques in strongly polynomial time $O(|V|^{2}\cdot\text{SFM}(|V|))$.

II-B Existing Results

In this section, we introduce the notation and two concepts relating to the minimum sum-rate problem: the Dilworth truncation and the principal sequence of partitions (PSP). We also present the coordinatewise saturation (CoordSat) algorithm, an essential nesting algorithm in the MDA algorithm in [6] for solving (2). The purpose is to review the existing results that are required to prove the strong map property in Section III.

II-B1 Preliminaries

For $X\subseteq V$, let $\chi_{X}=(r_{i}\colon i\in V)$ be the characteristic vector of the subset $X$ such that $r_{i}=1$ if $i\in X$ and $r_{i}=0$ if $i\notin X$. The notation $\chi_{\{i\}}$ is simplified by $\chi_{i}$. For $i\in V$, let $V_{i}=\{1,\dotsc,i\}$ be the set of the first $i$ users in $V$. Let $\sqcup$ denote the disjoint union. For $\mathcal{X}$ that contains disjoint subsets of $V$, we denote by $\tilde{\mathcal{X}}=\sqcup_{C\in\mathcal{X}}C$ the fusion of $\mathcal{X}$. For example, for $\mathcal{X}=\{\{3,4\},\{2\},\{8\}\}$, $\tilde{\mathcal{X}}=\{2,3,4,8\}$. For partitions $\mathcal{P},\mathcal{P}^{\prime}\in\Pi(V)$, we denote by $\mathcal{P}\preceq\mathcal{P}^{\prime}$ if $\mathcal{P}$ is finer than $\mathcal{P}^{\prime}$ and $\mathcal{P}\prec\mathcal{P}^{\prime}$ if $\mathcal{P}$ is strictly finer than $\mathcal{P}^{\prime}$.

Function $f\colon 2^{V}\mapsto\mathbb{R}$ is submodular if $f(X)+f(Y)\geq f(X\cap Y)+f(X\cup Y)$ for all $X,Y\subseteq V$. The problem $\min\{f(X)\colon X\subseteq V\}$ is a submodular function minimization (SFM) problem. It can be solved in strongly polynomial time and the set of minimizers $\arg\!\min\{f(X)\colon X\subseteq V\}$ form a set lattice such that the smallest minimizer $\bigcap\arg\!\min\{f(X)\colon X\subseteq V\}$ and largest minimizer $\bigcup\arg\!\min\{f(X)\colon X\subseteq V\}$ uniquely exist and can be determined at the same time when the SFM problem is solved [11, Chapter VI]. $P(f,\leq)=\{\mathbf{r}_{V}\colon r(X)\leq f(X),X\subseteq V\}$ and $B(f,\leq)=\{\mathbf{r}_{V}\in P(f,\leq)\colon r(V)=f(V)\}$ are the submodular polyhedron and base polyhedron, respectively. We call $f^{V_{i}}\colon 2^{V_{i}}\mapsto\mathbb{R}$ such that $f^{V_{i}}(X)=f(X)$ for all $X\subseteq V_{i}$ the reduction of $f$ on $V_{i}$ [11, Section 3.1(a)].

II-B2 Dilworth Truncation

For $\alpha\in\mathbb{R}_{+}$, define a set function $f_{\alpha}\colon 2^{V}\mapsto\mathbb{R}$ such that $f_{\alpha}(X)=\alpha-H(V)+H(X),\forall X\subseteq V$ except that $f(\emptyset)=0$. Let $f_{\alpha}[\cdot]$ be a partition function such that $f_{\alpha}[\mathcal{P}]=\sum_{C\in\mathcal{P}}f_{\alpha}(C)$ for all $\mathcal{P}\in\Pi(V)$. The Dilworth truncation of $f_{\alpha}$ is [22]

[TABLE]

Let $\mathcal{Q}_{\alpha,V}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V)}f_{\alpha}[\mathcal{P}]$ denote the finest minimizer of (3).555The minimizers of (3) form a partition lattice such that the finest and coarsest minimizers uniquely exist [8]. The solution to (3) exhibits a strong structure in $\alpha$ that is described by the PSP.

II-B3 Principal Sequence of Partitions (PSP)

The Dilworth truncation $\hat{f}_{\alpha}(V)$ is piecewise linear strictly increasing in $\alpha$. It is determined by $p<|V|$ critical points $\alpha^{(p)}<\dotsc<\alpha^{(1)}<\alpha^{(0)}=H(V)$ with the corresponding finest minimizer $\mathcal{P}^{(j)}=\mathcal{Q}_{\alpha^{(j)},V}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V)}f_{\alpha_{j}}[\mathcal{P}]$ for all $j\in\{0,\dotsc,p\}$ forming a partition chain called the PSP of the ground set $V$

[TABLE]

such that $\mathcal{Q}_{\alpha,V}=\mathcal{P}^{(j)}$ for all $\alpha\in(\alpha^{(j+1)},\alpha^{(j)}]$ [9, 8]. The first critical point of PSP provides the solutions to the minimum sum-rate problem [6, Corollary A.3]: $R_{\text{CO}}(V)=\alpha^{(1)}$ and $\mathcal{P}^{(1)}$ is the finest maximizer of (2).

II-B4 CoordSat Algorithm

In the MDA algorithm proposed in [5, 6], the main task is to solve the minimization problem (3) for a given value of $\alpha$ by calling the CoordSat algorithm in Algorithm 1. In step 1, the set function $g_{\alpha}$ is defined as

[TABLE]

where $r_{\alpha}(X)=\sum_{i\in X}r_{\alpha,i}$ is the sum-rate function of the rate vector $\mathbf{r}_{\alpha,V}=(r_{\alpha,i}\colon i\in V)$ for a given $\alpha$.

The CoordSat algorithm is based on the min-max relationship [11, Section 2.3][6, Lemma 23]

[TABLE]

where (5a) is a SFM problem [6, Section V-B] and the maximum of (5b) is called the saturation capacity. The idea is to saturate each dimension $i$ of a rate vector $\mathbf{r}_{\alpha,V}\in P(f_{\alpha},\leq)$ until it reaches the base polyhedron $B(\hat{f}_{\alpha},\leq)$ with $r_{\alpha}(V)=\hat{f}_{\alpha}(V)$ and $\mathcal{Q}_{\alpha,V}$ being updated to the finest minimizer of $\min_{\mathcal{P}\in\Pi(V)}f_{\alpha}[\mathcal{P}]$ [6, Section V-B]. Note, we use the notation $\mathcal{Q}_{\alpha,V_{i}}$ in Algorithm 1 since we will show in Section  III-A that $\mathcal{Q}_{\alpha,V_{i}}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V_{i})}f_{\alpha}[\mathcal{P}]$ after step 1 for all $i$.

Based on the outputs of the CoordSat algorithm and the properties of the PSP in Lemma 3 in Appendix A, the MDA algorithm proposed in [6] updates $\alpha$, the estimation of the minimum sum-rate $R_{\text{CO}}(V)$, towards the optimal one and finally returns an optimal rate vector $\mathbf{r}_{R_{\text{CO}}(V),V}\in B(\hat{f}_{R_{\text{CO}}(V)},\leq)$ and a partition $\mathcal{Q}_{R_{\text{CO}}(V),V}$ being the finest maximizer of (2) [6, Theorem 17].

III Parametric Approach

In this section, we reveal the structured properties of the partition $\mathcal{Q}_{\alpha,V_{i}}$ and rate vector $\mathbf{r}_{\alpha,V}$ in the CoordSat algorithm and propose a parametric (PAR) algorithm showing that, instead of running the CoordSat algorithm for a particular value of $\alpha$, we can obtain $\mathcal{Q}_{\alpha,V_{i}}$ and $\mathbf{r}_{\alpha,V}$ for all $\alpha$ in each iteration $i$.666In this paper, when we say for all $\alpha$, we mean for all $\alpha\in[0,H(V)]$ since the minimum sum-rate must be in the range $[0,H(V)]$. We will show in Section III-D that this PAR algorithm leads to a complexity reduction.

III-A Observations

For each $i$, we observe the values of $\mathcal{Q}_{\alpha,V_{i}}$ and $\mathbf{r}_{\alpha,V_{i}}$ in $\alpha$ in the CoordSat algorithm and have the following result.

Proposition 1

In each iteration $i$ of Algorithm 1, $\mathcal{Q}_{\alpha,V_{i}}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V_{i})}f_{\alpha}[\mathcal{P}]$ and $\mathbf{r}_{\alpha,V_{i}}\in B(\hat{f}_{\alpha}^{V_{i}},\leq)$ for all $\alpha$ after step 1.

Proof:

The proof is straightforward for that $\mathcal{Q}_{\alpha,V_{i}}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V_{i})}f_{\alpha}[\mathcal{P}]$ and $\mathbf{r}_{\alpha,V_{i}}\in B(\hat{f}_{\alpha}^{V_{i}},\leq)$ are returned by the call $\text{CoordSat}(\alpha,V_{i},H)$. ∎

Then, $\mathcal{Q}_{\alpha,V_{i}}$ for all $\alpha$ is again characterized by the PSP of $V_{i}$ with the number of critical points bounded by $|V_{i}|$. The function $g_{\alpha}$ in (4) is defined on $2^{\mathcal{Q}_{\alpha,V_{i}}}$ where $\mathcal{Q}_{\alpha,V_{i}}$ is a segmented partition variable in $\alpha$ and so is $\mathbf{r}_{\alpha,V_{i}}$.

Example 1

Consider a $5$-user system with

[TABLE]

where each $\mathsf{W}_{j}$ is an independent uniformly distributed random bit.

We call $\text{CoordSat}(\alpha,V,H)$ by setting $\alpha=3$. We initiate $\mathbf{r}_{3,V}\leftarrow(\alpha-H(V))\chi_{V}=(-7,\dotsc,-7)$, update $r_{3,1}=f_{3}(\{1\})=1$ and assign $\mathcal{Q}_{3,V_{1}}=\{\{1\}\}$. For $i=2$, we set $\mathcal{Q}_{3,V_{2}}=\{\{1\},\{2\}\}$ and consider the minimization problem $\min\{g_{3}(\tilde{\mathcal{X}})\colon\{2\}\in\mathcal{X}\subseteq\mathcal{Q}_{3,V_{2}}\}$. The minimal minimizer is $\mathcal{U}_{3,V_{2}}=\{\{2\}\}$. We do the updates $r_{3,2}=-7+g_{3}(\tilde{\mathcal{U}}_{3,V_{2}})=-1$ and \mathcal{Q}_{3,V_{2}}=\big{(}\{\{1\},\{2\}\}\setminus\mathcal{U}_{3,V_{2}}\big{)}\sqcup\{\tilde{\mathcal{U}}_{3,V_{2}}\}=\{\{1\},\{2\}\}. We stop here to consider another value of $\alpha$.

We call $\text{CoordSat}(\alpha,V,H)$ by setting $\alpha=6$. We initiate $\mathbf{r}_{6,V}\leftarrow(\alpha-H(V))\chi_{V}=(-4,\dotsc,-4)$ and set $r_{6,1}=f_{6}(\{1\})=4$ and $\mathcal{Q}_{6,V_{1}}=\{\{1\}\}$. We have $\mathcal{U}_{6,V_{2}}=\{\{1\},\{2\}\}=\bigcap\arg\!\min\{g_{6}(\tilde{\mathcal{X}})\colon\{2\}\in\mathcal{X}\subseteq\mathcal{Q}_{6,V_{2}}\}$ and do the updates $r_{6,2}=-4+g_{6}(\tilde{\mathcal{U}}_{6,V_{2}})=0$ and \mathcal{Q}_{6,V_{2}}=\big{(}\{\{1\},\{2\}\}\setminus\mathcal{U}_{6,V_{2}}\big{)}\sqcup\{\tilde{\mathcal{U}}_{6,V_{2}}\}=\{\{1,2\}\}.

One can show that $\{\{1\},\{2\}\}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V_{2})}f_{3}[\mathcal{P}]$ and $\{\{1,2\}\}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V_{2})}f_{6}[\mathcal{P}]$. In fact, repeating the above procedure for all $\alpha$, we have the segmented $\mathbf{r}_{\alpha,\{1,2\}}$ and $\mathcal{Q}_{\alpha,V_{2}}$

[TABLE]

because of the segmented $\tilde{\mathcal{U}}_{\alpha,V_{2}}$

[TABLE]

Proposition 1 suggests that, in each iteration of the CoordSat algorithm, we can obtain $\mathbf{r}_{\alpha,V_{i}}$ and $\mathcal{Q}_{\alpha,V_{i}}$ for all values of $\alpha$, which in fact completes the task of determining the PSP of $V_{i}$ (see Section III-D1). To do so, it is essential to discuss how to determine $\tilde{\mathcal{U}}_{\alpha,i}$ for all $\alpha$. It should also be noted that we automatically know $\mathcal{U}_{\alpha,V_{i}}$ if $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ is obtained in that $\mathcal{U}_{\alpha,V_{i}}=\{C\in\mathcal{Q}_{\alpha,V_{i}}\colon C\subseteq\tilde{\mathcal{U}}_{\alpha,V_{i}}\}$.777This means that $\mathcal{U}_{\alpha,V_{i}}$ is the decomposition of $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ by $\mathcal{Q}_{\alpha,V_{i}}$. Here, we should use the value of $\mathcal{Q}_{\alpha,V_{i}}$ in the minimization problem $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{i\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{i}}\}$ before the updates in step 1. For example, for $\tilde{\mathcal{U}}_{\alpha,V_{2}}$ in (7),

[TABLE]

III-B Strong Map Property

If $\mathcal{Q}_{\alpha,V_{i}}=\bigwedge\arg\!\min_{\mathcal{P}\in\Pi(V_{i})}f_{\alpha}[\mathcal{P}]$ after the update $\mathcal{Q}_{\alpha,V_{i}}\leftarrow(\mathcal{Q}_{\alpha,V_{i}}\setminus\mathcal{U}_{\alpha,V_{i}})\sqcup\{\tilde{\mathcal{U}}_{\alpha,V_{i}}\}$ in step 1 in Algorithm 1 based on Proposition 1, $\mathcal{Q}_{\alpha,V_{i}}$ must satisfy the properties of the PSP in Section II-B3: $\mathcal{Q}_{\alpha,V_{i}}$ gets monotonically coarser in $\alpha$ and is characterized by a finite number of critical points.888$\mathcal{Q}_{\alpha,V_{i}}$ is in fact the PSP of $V_{i}$ with an offset in $\alpha$. See Section III-D1. It necessarily means the we must have $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ being segmented and the size of $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ must increase in $\alpha$. This observations can be justified by the strong map property of the function $g_{\alpha}$, which also states that all critical points that characterize the segmented $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ can be determined by the parametric submodular function minimization (PSFM) algorithms.

Definition 1** (strong map [23, Section 4.1])**

For two distributive lattices $\mathcal{L}_{1},\mathcal{L}_{2}\subseteq 2^{V}$,999A group of sets $\mathcal{L}$ form a distributive lattice if, for all $X,Y\in\mathcal{L}$, $X\cap Y\in\mathcal{L}$ and $X\cup Y\in\mathcal{L}$ [11, Section 3.2]. and submodular functions $f_{1}\colon\mathcal{L}_{1}\mapsto\mathbb{R}$ and $f_{2}\colon\mathcal{L}_{2}\mapsto\mathbb{R}$, $f_{1}$ and $f_{2}$ form a strong map, denoted by $f_{1}\rightarrow f_{2}$, if

[TABLE]

for all $X,Y\in\mathcal{L}_{1}\cap\mathcal{L}_{2}$ such that $X\subseteq Y$. The strong map is strict, denoted by $f_{1}\twoheadrightarrow f_{2}$, if $f_{1}(Y)-f_{1}(X)>f_{2}(Y)-f_{2}(X)$ for all $X\subsetneq Y$.

Theorem 1

In each iteration $i$ of the CoordSat algorithm, $g_{\alpha}$ forms a strict strong map in $\alpha$:

[TABLE]

Proof:

For any $\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{i}}$ and $\mathcal{Y}\subseteq\mathcal{Q}_{\alpha^{\prime},V_{i}}$ such that $\{i\}\in\tilde{\mathcal{X}}\subseteq\tilde{\mathcal{Y}}$ and $g_{\alpha}$ and $g_{\alpha^{\prime}}$ are both defined on $\tilde{\mathcal{X}}$ and $\tilde{\mathcal{Y}}$, we have $i\notin\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}}$. Also, there exist $\mathcal{M}\subseteq\mathcal{Q}_{\alpha,V_{i}}$ and $\mathcal{N}\subseteq\mathcal{Q}_{\alpha^{\prime},V_{i}}$ such that $\tilde{\mathcal{M}}=\tilde{\mathcal{N}}=\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}}$ (with $\mathcal{M}\preceq\mathcal{N}$), $r_{\alpha}(\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}})=f_{\alpha}[\mathcal{M}]=\hat{f}_{\alpha}(\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}})$ and $r_{\alpha^{\prime}}(\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}})=f_{\alpha^{\prime}}[\mathcal{N}]=\hat{f}_{\alpha^{\prime}}(\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}})$.101010The following holds for all $\alpha$ and $i$: (a) $\hat{f}_{\alpha}(\tilde{\mathcal{X}})=\hat{f}_{\alpha}[X]$ for all $\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{i}}$ [6, Theorem 38 and Lemma 39]; (b) $\hat{f}_{\alpha}(C)=f_{\alpha}(C)=r_{\alpha}(C)$ for all $C\in\mathcal{Q}_{\alpha,V_{i}}$ [6, proof of Theorem 38]. Then,

[TABLE]

where $\hat{f}_{\alpha^{\prime}}(\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}})-\hat{f}_{\alpha}(\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}})>0$ for all $\alpha$ and $\alpha^{\prime}$ such that $\alpha<\alpha^{\prime}$ since $\hat{f}_{\alpha}(\tilde{\mathcal{Y}}\setminus\tilde{\mathcal{X}})$ is strictly increasing in $\alpha$ (see Section II-B3). ∎

The strict strong map property directly leads to the structured property of $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ in $\alpha$ based on the results in [23].

Lemma 1

[23, Theorems 26 to 28]** In each iteration of the CoordSat algorithm, the minimal minimizer $\mathcal{U}_{\alpha,V_{i}}$ of $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{i\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{i}}\}$ satisfies $\tilde{\mathcal{U}}_{\alpha,V_{i}}\subseteq\tilde{\mathcal{U}}_{\alpha^{\prime},V_{i}}$ for all $\alpha<\alpha^{\prime}$. In addition, $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ for all $\alpha$ is fully characterized by $q<|V_{i}|-1$ critical points $0\leq\alpha_{q}<\dotsc<\alpha_{1}<\alpha_{0}=H(V)$ and the corresponding $\tilde{{\mathcal{S}}}_{j}=\tilde{\mathcal{U}}_{\alpha_{j},V_{i}}$ for all $j\in\{0,\dotsc,q\}$ forms a set chain

[TABLE]

such that $\tilde{\mathcal{U}}_{\alpha,V_{i}}=\tilde{{\mathcal{S}}}_{q}=\{i\}$ for all $\alpha\in[0,\alpha_{q}]$ and $\tilde{\mathcal{U}}_{\alpha,V_{i}}=\tilde{{\mathcal{S}}}_{j}$ for all $\alpha\in(\alpha_{j+1},\alpha_{j}]$ such that $j\in\{0,\dotsc,q-1\}$.111111It should be noted that the value of $\alpha^{(j)}$s in the PSP and $\alpha_{j}$s in Lemma 1 do not necessarily coincide and the critical points $\alpha_{j}$s are different for $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{i\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{i}}\}$ with a different $i$. ∎

Example 2

In Example 1, we have $\tilde{\mathcal{U}}_{\alpha,V_{2}}$ in (7) characterized by the critical points $\alpha_{1}=4$ and $\alpha_{0}=H(V)=10$ with $\tilde{{\mathcal{S}}}_{1}=\{2\}$ and $\tilde{{\mathcal{S}}}_{0}=\{1,2\}$ such that $\{2\}=\tilde{{\mathcal{S}}}_{1}\subsetneq\tilde{{\mathcal{S}}}_{0}=V_{2}$, $\tilde{\mathcal{U}}_{\alpha,V_{2}}=\tilde{{\mathcal{S}}}_{1}$ for $\alpha\in[0,\alpha_{1}]$ and $\tilde{\mathcal{U}}_{\alpha,V_{2}}=\tilde{{\mathcal{S}}}_{0}$ for $\alpha\in(\alpha_{1},\alpha_{0}]$.

We continue the procedure in Example 1 for $i=3$ by considering the problem $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{3\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{2}}\sqcup\{\{3\}\}\}$ where $\mathcal{Q}_{\alpha,V_{2}}$ and $\mathbf{r}_{\alpha,V_{2}}$ are in (6). We have

[TABLE]

that is determined by the critical points $\alpha_{1}=8$ and $\alpha_{0}=H(V)=10$ with $\tilde{{\mathcal{S}}}_{1}=\{3\}$ and $\tilde{{\mathcal{S}}}_{0}=\{1,2,3\}$ such that $\{3\}=\tilde{{\mathcal{S}}}_{1}\subsetneq\tilde{{\mathcal{S}}}_{0}=V_{3}$. After the updates in step 1, we have

[TABLE]

III-C Parametric Method

Lemma 1 directly suggests the PAR algorithm in Algorithm 2, where, in each iteration $i$, we determine the values of $\mathcal{Q}_{\alpha,V_{i}}$ and $\mathbf{r}_{\alpha,V_{i}}$ for all $\alpha$.121212We call Algorithm 2 a parametric algorithm since the variables $\mathcal{Q}_{\alpha,V_{i}}$ and $\mathbf{r}_{\alpha,V_{i}}$ in each iteration $i$ are parameterized by $\alpha$ and $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ for all $\alpha$ can be determined by a PSFM algorithm that is parameterized by $\alpha$.

Example 3

We apply the PAR algorithm to the system in Example 1. We first initiate $r_{\alpha,i}=\alpha-H(V))=\alpha-10$ for all $i$ and $\alpha$. For $i=1$, we have $\mathcal{Q}_{\alpha,V_{1}}=\{\{1\}\}$ and $r_{\alpha,1}=f_{\alpha}(\{1\})=\alpha-2$ for all $\alpha$. See Fig. 1(a).

As shown in Example 2, for $i=2$ and $i=3$, we have $\tilde{\mathcal{U}}_{\alpha,V_{2}}$ in (7) so that the updated $\mathbf{r}_{\alpha,V_{2}}$ and $\mathcal{Q}_{\alpha,V_{2}}$ are in (6) and $\tilde{\mathcal{U}}_{\alpha,V_{3}}$ in (9) so that the updated $\mathbf{r}_{\alpha,V_{3}}$ and $\mathcal{Q}_{\alpha,V_{3}}$ are in (10), respectively. See (b) and (c) in Fig. 1.

For $i=4$, consider the problem $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{4\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{4}}\}$ where $\mathcal{Q}_{\alpha,V_{4}}=\mathcal{Q}_{\alpha,V_{3}}\sqcup\{\{4\}\}$. We have the critical points $\alpha_{1}=7$ and $\alpha_{0}=H(V)=10$ with $\tilde{{\mathcal{S}}}_{1}=\{4\}$ and $\tilde{{\mathcal{S}}}_{0}=\{1,2,3,4\}$ such that $\{4\}=\tilde{{\mathcal{S}}}_{1}\subsetneq\tilde{{\mathcal{S}}}_{0}=V_{4}$ and

[TABLE]

We use $\tilde{\mathcal{U}}_{\alpha,V_{4}}$ to update $\mathbf{r}_{\alpha,V}$ and $\mathcal{Q}_{\alpha,V_{4}}$ for all $\alpha$ as in step 2 and get

[TABLE]

See Fig. 1(d).

For $i=5$, we have the critical points for the problem $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{5\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V}\}$ being $\alpha_{2}=6$ , $\alpha_{1}=6.5$ and $\alpha_{0}=H(V)=10$ with $\tilde{{\mathcal{S}}}_{2}=\{5\}$, $\tilde{{\mathcal{S}}}_{1}=\{1,2,5\}$ and $\tilde{{\mathcal{S}}}_{0}=\{1,\dotsc,5\}$ such that

[TABLE]

After the updates in step 2, we have

[TABLE]

See Fig. 1(e). Here, we finally update to $\mathcal{Q}_{\alpha,V}$ for all $\alpha$. The corresponding PSP has the critical points $\alpha^{(3)}=4$, $\alpha^{(2)}=6$ and $\alpha^{(1)}=6.5$ and $\alpha^{(0)}=H(V)=10$ with $\mathcal{P}^{(3)}=\{\{1\},\dotsc,\{5\}\}$, $\mathcal{P}^{(2)}=\{\{1,2\},\{3\},\{4\},\{5\}\}$, $\mathcal{P}^{(1)}=\{\{1,2,5\},\{3\},\{4\}\}$ and $\mathcal{P}^{(0)}=\{\{1,\dotsc,5\}\}$ so that we know $R_{\text{CO}}(V)=\alpha^{(1)}=6.5$ is the minimum sum-rate and $\mathcal{P}^{(1)}=\{\{1,2,5\},\{3\},\{4\}\}$ is finest maximizer of problem (2). We also know an optimal achievable rate vector $\mathbf{r}_{6.5,V}=(4.5,0,0.5,0.5,1)$.

The rest of problem is how to obtain the critical points $\alpha_{j}$s and $\tilde{{\mathcal{S}}}_{j}$s in Lemma 1. Since $\mathcal{Q}_{\alpha,V_{i}}$ for all $\alpha$ determines all partitions in the PSP of $V_{i}$ (see Section III-D1), we can still use Lemma 3 in Appendix A to adapt the value of $\alpha$ so that all $\tilde{{\mathcal{S}}}_{j}$s are determined by the call $\text{StrMap}(\{\{m\}\colon m\in V_{i}\},\{V_{i}\})$ in Algorithm 3, and the corresponding $\alpha_{j}$s can be obtained by another property of the strict strong map property below.

Lemma 2** ([23, Theorem 31])**

For all $\alpha_{j}$s and $\tilde{{\mathcal{S}}}_{j}$s that characterize $\tilde{\mathcal{U}}_{\alpha,V_{i}}$ of the minimal minimizer of $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{i\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{i}}\}$ in Lemma 1,

[TABLE]

Example 4

We call $\text{StrMap}(\{\{1\},\dotsc,\{5\}\},\{\{1,\dotsc,5\}\})$ to determine all $\tilde{{\mathcal{S}}}_{j}$s for $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{5\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V}\}$ at the iteration $i=5$ in Example 3. Here, $\mathcal{Q}_{\alpha,V}=\mathcal{Q}_{\alpha,V_{4}}\sqcup\{\{5\}\}$ where $\mathcal{Q}_{\alpha,V_{4}}$ is in (11). In this call, we have $\alpha=5.75$ and $\mathcal{S}=\bigcap\arg\!\min\{g_{5.75}(\tilde{\mathcal{X}})\colon\{5\}\in\mathcal{X}\subseteq\mathcal{Q}_{5.75,V}\}=\{\{1,2\},\{5\}\}$ so that $\mathcal{P}=(\mathcal{Q}_{5.75,V}\setminus\mathcal{S})\sqcup\{\tilde{\mathcal{S}}\}=\{\{1,2,5\},\{3\},\{4\}\}$. Since $\mathcal{P}\neq\{\{1\},\dotsc,\{5\}\}$, recursion continues.

In the call $\text{StrMap}(\{\{1\},\dotsc,\{5\}\},\{\{1,2,5\},\{3\},\{4\}\})$, we have $\alpha=5$, $\mathcal{S}=\{\{5\}\}$ and $\mathcal{P}=\{\{1,2\},\{3\},\{4\},\{5\}\}$. We call $\text{StrMap}(\{\{1\},\dotsc,\{5\}\},\{\{1,2\},\{3\},\{4\},\{5\}\})$ and $\text{StrMap}(\{\{1,2\},\{3\},\{4\},\{5\}\},\{\{1,2,5\},\{3\},\{4\}\})$ and get $\{5\}$ returned for both calls. In the call $\text{StrMap}(\{\{1,2,5\},\{3\},\{4\}\},\{\{1,\dotsc,5\}\})$, we have $\alpha=6.5$, $\mathcal{S}=\{\{1,2\},\{5\}\}$ and $\mathcal{P}=\{\{1,2,5\},\{3\},\{4\}\}$ so that it terminates with $\{1,2,5\}$ returned.

Finally we have $\{5\}$ and $\{1,2,5\}$ as $\tilde{{\mathcal{S}}}_{2}$ and $\tilde{{\mathcal{S}}}_{1}$, respectively, and know that $\tilde{{\mathcal{S}}}_{0}=V=\{1,\dotsc,5\}$. We apply Lemma 2 to determine that $\alpha_{2}=6$, $\alpha_{1}=6.5$ and $\alpha_{0}=H(V)=10$.

III-D Complexity

Due to the strong map property in Theorem 1, the StrMap algorithm can be completed by a parametric submodular function minimization (PSFM) algorithm, e.g., [15, 16, 17], that runs in the same asymptotic time as a single call of a SFM algorithm. See Appendix B. Therefore, the minimum sum-rate problem can be solved by the PAR algorithm in $O(|V|\cdot\text{SFM}(|V|))$ time. As compared to $O(|V|^{2}\cdot\text{SFM}(|V|))$ time of the MDA algorithm in [6], the complexity is reduced by a factor of $|V|$.

III-D1 CO in Expanding Ground Set

An observation on Fig. 1, the minimum sum-rate problem can be solved when the size of the ground set $V_{i}$ is gradually increasing in the order of $i=1,2,\dotsc,|V|$.131313This is particularly useful when the users complete recording their observations in different times. For example $i=1$ is assigned to the user that completes observations first. In addition, by replacing the horizontal axis by $\alpha\leftarrow\alpha-H(V)+H(V_{i})$ in Figs. 1(a)-(e), we have them being exactly the PSP of $V_{i}$ that provides the solution to the minimum sum-rate problem in $V_{i}$. For example, in Fig. 1(b), by letting $\alpha\leftarrow\alpha-H(V)+H(V_{2})=\alpha-2$, we have $\mathcal{Q}_{\alpha,V_{2}}$ determining the PSP of $V_{2}$ so that the first critical point $\alpha^{(1)}=2=R_{\text{CO}}(V_{2})$. See Fig. 2.

III-D2 Distributed Implementation

The PAR algorithm can be implemented in a distributed manner: for all $i=1,\dotsc,|V|$, let user $i$ obtain $\mathcal{Q}_{\alpha,V_{i}}$ and $\mathbf{r}_{\alpha,V_{i}}$ for all $\alpha$ and pass them to user $i+1$.

IV Successive Omniscience

Successive omniscience (SO) is proposed in [12, 13]. The idea is based on the fact that there is a particular group of nonsingleton subsets $X^{*}$s of $V$ such that the local omniscience in $X^{*}$ can be attained with the minimum sum-rate $R_{\text{CO}}(X^{*})$ before the global omniscience in $V$ so that the overall sum-rate for the CO problem still remains the minimum $R_{\text{CO}}(V)$. We call $X^{*}$ a complimentary subset (for SO) [12]. The existence of a complimentary subset suggests that omniscience can be attained in a successive manner: first select and attain local omniscience in $X^{*}$; then solve the global omniscience problem in $V$. An example in [14, Section IV] shows that recursively applying this procedure leads to a multi-stage SO towards the global omniscience.

Obviously, the main task in SO is to select a subset $X^{*}$ that is complimentary. It is shown in [12] that a non-singleton $X^{*}\subsetneq V$ is complimentary if and only if $H(V)-H(X^{*})+R_{\text{CO}}(X^{*})\leq R_{\text{CO}}(V)$. The interpretation is that: after local omniscience in $X^{*}$ is attained by minimum sum-rate $R_{\text{CO}}(X^{*})$, the other users in $V\setminus X^{*}$ should transmit at least the missing randomness $H(V)-H(X^{*})$ to attain global omniscience; the overall rate $H(V)-H(X^{*})+R_{\text{CO}}(X^{*})$ must be no greater than $R_{\text{CO}}(V)$ for $X^{*}$ to be complimentary. This necessary and sufficient condition is shown to be equivalent to $f_{R_{\text{CO}}(V)}(X^{*})=\hat{f}_{R_{\text{CO}}(V)}(X^{*})$ [14, Corollary III.3]. The authors in [14] further relaxed it to a sufficient condition based on a lower bound on $R_{\text{CO}}(V)$ as follows.

Proposition 2** ([14, Lemma III.7])**

A nonsingleton $X^{*}\subsetneq V$ is complimentary if $f_{\underline{\alpha}}(X^{*})=\hat{f}_{\underline{\alpha}}(X^{*})$ for $\underline{\alpha}\leq R_{\text{CO}}(V)$. ∎

However, to implement SO, we still need to know an optimal rate vector for attaining the local omniscience in $X^{*}$ with sum-rate $R_{\text{CO}}(X^{*})$, which is not addressed in [6] or the existing literature. One may think of a two step approach: choose a $X^{*}$; solve the minimum sum-rate problem in $X^{*}$ by the existing CO techniques, e.g., the MDA algorithm in [6]. But, it is worth discussing if there exists a less complex approach: can an optimal rate vector for the local omniscience in $X^{*}$ be obtained at the same time when $X^{*}$ is selected? The following theorem shows that we can do so by utilizing the results in the PAR algorithm. The proof is in Appendix C.

Theorem 2

Let $\mathcal{Q}_{\alpha,V_{i}}$ and $\mathbf{r}_{\alpha,V_{i}}$ be segmented partition and rate vector, respectively, obtained at the end of iteration $i\in\{2,\dotsc,|V|\}$ of the PAR algorithm. For $\underline{\alpha}=\sum_{i\in V}\frac{H(V)-H(\{i\})}{|V|-1}$, all $C\in\mathcal{Q}_{\underline{\alpha},V_{i}}$ such that $|C|>1$ satisfy $f_{\underline{\alpha}}(C)=\hat{f}_{\underline{\alpha}}(C)$, i.e., they are complimentary subsets. Also, let $\alpha^{*}$ be the minimum value of $\alpha$ such that $f_{\alpha}(C)=\hat{f}_{\alpha}(C)$. $\mathbf{r}_{\alpha^{*},C}$ is an optimal rate vector that attains local omniscience in $C$ with the minimum sum-rate $R_{\text{CO}}(C)$. ∎

Theorem 2 suggests that a complimentary subset for SO, if there exists any, can be searched at the end of each iteration of the PAR algorithm. Therefore, we have the SO algorithm in Algorithm 4 that completes in $O(|V|\cdot\text{SFM}(|V|))$ time. In addition, $\alpha^{*}\leq\underline{\alpha}$ always and $\alpha^{*}$ is not hard to determine from $\mathcal{Q}_{\alpha,V_{i}}$: It is the critical value or turning point where all subsets of $C$ merges to $C$. See the example below.

Example 5

We apply Theorem 2 to the system in Example 1. We set $\underline{\alpha}=\sum_{i\in V}\frac{H(V)-H(\{i\})}{|V|-1}=5.75$. For $i=2$, we obtain the $\mathcal{Q}_{\alpha,V_{2}}$ and $\mathbf{r}_{\alpha,V_{2}}$ for all $\alpha$ as in (6) at the end of the iteration $i=2$ in the PAR algorithm so that $\mathcal{Q}_{\underline{\alpha},V_{2}}=\{\{1,2\}\}$. The only nonsingleton $\{1,2\}$ is a complimentary subset. We then search region $[0,\underline{\alpha}]$ and find that $\{1\}$ and $\{2\}$ merge to $\{1,2\}$ at $\alpha^{*}=4$. See Fig. 3. Note, this meas $\alpha^{*}=\min\{\alpha\colon f_{\alpha}(\{1,2\})=\hat{f}_{\alpha}(\{1,2\})\}=4$, where we have $\mathbf{r}_{4,V_{2}}=(2,0)$ being an optimal rate vector that attains local omniscience in $\{1,2\}$ with $R_{\text{CO}}(\{1,2\})=r_{4}(\{1,2\})=2$. For the optimal rate vector $(4.5,0,0.5,0.5,1)$ for the global omniscience obtained in Example 3, we have $(4.5,0,0.5,0.5,1)=(2,0,0,0,0)+(2.5,0,0.5,0.5,1)$, which means that, by letting users transmit at rate $(2.5,0,0.5,0.5,1)$ after the local omniscience in $\{1,2\}$, the global omniscience is still attained with the minimum sum-rate $R_{\text{CO}}(V)$.

Also note that Theorem 2 in fact applies to any lower bound $\underline{\alpha}\leq R_{\text{CO}}(V)$. For example, for $\underline{\alpha}=6.25$, consider $\mathcal{Q}_{\alpha,V}$ obtained by the PAR algorithm for $i=5$ in Fig. 1(e). We have $\{1,2,5\}\in\mathcal{Q}_{6.25,V}$ being another complimentary subset with $\alpha^{*}=\min\{\alpha\colon f_{\alpha}(\{1,2,5\})=\hat{f}_{\alpha}(\{1,2,5\})\}=6$ and $\mathbf{r}_{6,\{1,2,5\}}=(4,0,1)$ being an optimal rate vector for attaining the local omniscience in $\{1,2,5\}$.

V Conclusion

We reduced the complexity of solving the minimum sum-rate problem in CO from $O(|V|^{2}\cdot\text{SFM}(|V|))$ to $O(|V|\cdot\text{SFM}(|V|))$ by proposing a PAR algorithm. We proved the strict strong map property, which ensures that the partitions and the rate vectors obtained in the existing CoordSat algorithm are segmented in $\alpha$, the estimation of the minimum sum-rate. We proposed the PAR algorithm where the critical points for the segmented variables can be searched by the PSFM algorithm so that the overall complexity reduces to $O(|V|\cdot\text{SFM}(|V|))$. We discussed how to apply the PAR algorithm to a growing ground set in a distributed manner. We also showed how to determine a complimentary user subset for SO and an optimal rate vector for attaining the local omniscience in it in $O(|V|\cdot\text{SFM}(|V|))$ time.

There is a potential that the PAR algorithm can be adapted so that the PSP is obtained in a growing ground set $V_{i}$ in a fully distributed manner. Note, in the distributed implementation in Section III-D2, the users still need to know the overall information $H(V)$ of the multiple source. Also, there should be more discussion on how to apply the PAR algorithm in a multi-stage SO.

Appendix A

Let $H[\mathcal{P}]=\sum_{C\in\mathcal{P}}H(C)$ for all $\mathcal{P}\in\Pi(V)$. The MDA algorithm proposed in [6] starts with $\mathcal{P}=\{\{i\}\colon i\in V\}$ and run the recursion $(\mathbf{r}_{V},\mathcal{P})\coloneqq\text{CoordSat}(\alpha,V,H)$ where $\alpha=H(V)-\frac{H[\mathcal{P}]-H(V)}{|\mathcal{P}|-1}$ until $\alpha$ converges to $R_{\text{CO}}(V)$. The validity of the MDA algorithm is based on the properties of the PSP below. Note, Lemma 3 also ensures the validity of StrMap algorithm in Algorithm 3.

Lemma 3** ([9, Sections 2.2 and 3][8, Definition 3.8])**

The $\alpha^{(j)}$s and $\mathcal{P}^{(j)}$s in the PSP of the ground set $V$ satisfy the followings.

• •

For all $j$, $\alpha^{(j)}=H(V)-\frac{H[\mathcal{P}^{(j)}]-H[\mathcal{P}^{(j-1)}]}{|\mathcal{P}^{(j)}|-|\mathcal{P}^{(j-1)}|}$;

• •

For all $j,j^{\prime}$ such that $j+1<j^{\prime}$, let $\alpha=H(V)-\frac{H[\mathcal{P}^{(j^{\prime})}]-H[\mathcal{P}^{(j)}]}{|\mathcal{P}^{(j^{\prime})}|-|\mathcal{P}^{(j)}|}$. Then, $\alpha^{(j)}<\alpha\leq\alpha^{(j^{\prime})}$

Appendix B

In [24], the push-relabel max-flow algorithm [25] was extended to the one that is parameterized by a parameter $\alpha$. The same technique was further applied to extend the SFM algorithms to the PSFM ones in [15, 16, 17]. For example, the PSFM algorithm proposed in [15] nests the Schrijver’s SFM algorithm [26] in a push-relabel framework so that, for the function $g_{\alpha}$ that forms strong map sequence in $\alpha$, all minimizers of $\min\{g_{\alpha}(\tilde{\mathcal{X}})\colon\{i\}\in\mathcal{X}\subseteq\mathcal{Q}_{\alpha,V_{i}}\}$ of decreasing or increasing values of $\alpha$ can be determined in the same asymptotic time as the Schrijver’s algorithm. See [15, Section 4.2] for the details.

Appendix C

Proof:

In Theorem 2, we have $\underline{\alpha}=\sum_{i\in V}\frac{H(V)-H(\{i\})}{|V|-1}\leq R_{\text{CO}}(V)$ based on (2). We prove $f_{\underline{\alpha}}(C)=\hat{f}_{\underline{\alpha}}(C)$ for all $C\in\mathcal{Q}_{\underline{\alpha},V_{i}}$ such that $|C|>1$ by contradiction. Assume that $f_{\underline{\alpha}}(C)\neq\hat{f}_{\underline{\alpha}}(C)$. Then, there must exist some $\mathcal{P}\in\Pi(C)\setminus\{C\}$ such that $f_{\underline{\alpha}}(C)>\hat{f}_{\underline{\alpha}}[\mathcal{P}]$ so that $f_{\underline{\alpha}}[\mathcal{Q}_{\underline{\alpha},V_{i}}]>f_{\underline{\alpha}}[\mathcal{P}]+\sum_{C^{\prime}\in\mathcal{Q}_{\underline{\alpha},V_{i}}\colon C^{\prime}\neq C}f_{\alpha}(C^{\prime})$. This contradicts $\mathcal{Q}_{\underline{\alpha},V_{i}}\in\arg\!\min_{\mathcal{P}\in\Pi(V_{i})}f_{\underline{\alpha}}[\mathcal{P}]$ in Proposition 1. Therefore, we must have $f_{\underline{\alpha}}(C)=\hat{f}_{\underline{\alpha}}(C)$. So, based on Proposition 2, all nonsingleton $C\in\mathcal{Q}_{\underline{\alpha},V_{i}}$ are complimentary.

For $\alpha^{*}=\min\{\alpha\colon f_{\alpha}(C)=\hat{f}_{\alpha}(C)\}$, we have $\alpha^{*}=H(V)-H(C)+\max_{\mathcal{P}\in\Pi(C)\colon|\mathcal{P}|>1}\sum_{X\in\mathcal{P}}\frac{H(C)-H(X)}{|\mathcal{P}|-1}=H(V)-H(C)+R_{\text{CO}}(C)$. Also, $\mathbf{r}_{\alpha^{*},C}\in B(\hat{f}_{\alpha^{*}}^{C},\leq)$ where $B(\hat{f}_{\alpha^{*}}^{C},\leq)=\{\mathbf{r}_{\alpha^{*},C}\in P(f_{\alpha^{*}}^{C},\leq)\colon r_{\alpha^{*}}(C)=\hat{f}_{\alpha^{*}}(C)=f_{\alpha^{*}}(C)\}$.141414Here, we use the property $P(f_{\alpha^{*}}^{C},\leq)=P(\hat{f}_{\alpha^{*}}^{C},\leq)$ [11, Theorems 2.5(i) and 2.6(i)]. We have $r_{\alpha^{*}}(X)\leq f_{\alpha^{*}}(X)=H(X)+\alpha^{*}-H(V)=H(X)+R_{\text{CO}}(C)-H(C)$ for all $X\subseteq C$ in $P(f_{\alpha^{*}}^{C},\leq)$. These constraints ensure that $\mathbf{r}_{\alpha^{*},C}\in\mathscr{R}(C)$. We also have $r_{\alpha^{*}}(C)=f_{\alpha^{*}}(C)=H(C)+\alpha^{*}-H(V)=R_{\text{CO}}(C)$. Therefore, $\mathbf{r}_{\alpha^{*},C}$ is an optimal rate vector for the local omniscience in $C$. ∎