The Hardness of Optimization Problems on the Weighted Massively Parallel Computation Model

TL;DR

This paper investigates the computational complexity of minimizing communication costs in a weighted network model for parallel computation, establishing hardness results for various problem variants.

Contribution

It introduces the Weighted Massively Parallel Computation (WMPC) model and characterizes the complexity of related communication minimization problems.

Findings

Some problems are in P

Some are NP-complete

Some are W[1]-complete

Abstract

The topology-aware Massively Parallel Computation (MPC) model is proposed and studied recently, which enhances the classical MPC model by the awareness of network topology. The work of Hu et al. on topology-aware MPC model considers only the tree topology. In this paper a more general case is considered, where the underlying network is a weighted complete graph. We then call this model as Weighted Massively Parallel Computation (WMPC) model, and study the problem of minimizing communication cost under it. Two communication cost minimization problems are defined based on different pattern of communication, which are the Data Redistribution Problem and Data Allocation Problem. We also define four kinds of objective functions for communication cost, which consider the total cost, bottleneck cost, maximum of send and receive cost, and summation of send and receive cost, respectively. Combining the two problems in different communication pattern with the four kinds of objective cost functions, 8 problems are obtained. The hardness results of the 8 problems make up the content of this paper. With rigorous proof, we prove that some of the 8 problems are in P, some FPT, some NP-complete, and some W[1]-complete.

Tables

Table 1. Table 1 : Summary of results

	TOTAL	BTNK	MSR	SSR
DRP	P (Section 2.1)	P (Section 2.2)	NP-complete (Section 2.3)	NP-complete (Section 2.4)
DAP-Cont	FPT (Section 3.2)	FPT (Section 3.2)	W[1]-complete (Section 3.3)	W[1]-complete (Section 3.3)

Equations

$$\underset{\pi \in \Pi (n)}{\min }\sum _{i=1}^n\sum _{j=1}^{n}T[i,j]C[i,{\pi }_j]$$

$$\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }\sum _{j=1}^{n}T[j,{\pi }_i^{-1}]C[j,i]$$

$$\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }\left\{\sum _{j=1}^{n}T[i,j]C[i,{\pi }_j],\sum _{j=1}^{n}T[j,{\pi }_i^{-1}]C[j,i]\right\}$$

$$\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }\left\{\sum _{j=1}^{n}T[i,j]C[i,{\pi }_j]+\sum _{j=1}^{n}T[j,{\pi }_i^{-1}]C[j,i]\right\}$$

$$\min$$

$$s.t.$$

$$\sum _{k=1}^n\sum _{i=1}^n\sum _{j=1}^{n}T[i,j]C[i,k]x_{jk}=\sum _{j=1}^n\sum _{k=1}^n\sum _{i=1}^{n}T[i,j]C[i,k]x_{jk}$$

$$=$$

$$\min$$

$$s.t.$$

$$T[i,j]=\{\begin{array}{rl}{s}_j,& if(i=1andj\in [1,n])or(i=2andj\in [1,n])\\ 0,& otherwise\end{array}$$

$$C[i,j]=\{\begin{array}{rl}1,& if(i=1,j\in [3,n+2])or(i=2,j\in [n+3,2n+2])\\ \Delta ,& if(i=1,j=2)or(i=2,j=1)\\ 0,& otherwise\end{array}$$

$$C[i,j]=\{\begin{array}{rl}1,& if(i\equiv j)modkandi\ne j\\ 0,& otherwise\end{array}$$

$$sen{d}_i=s_{j_2}+s_{j_3},$$

$$sen{d}_{k+i}=s_{j_1}+s_{j_3},$$

$$sen{d}_{2k+i}=0,$$

$$\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\sum _{i=1}^n\sum _{j=1}^{n}T[i,j]C[i,{\pi }_j]$$

$$\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }\sum _{j=1}^{n}T[j,{\pi }_i^{-1}]C[j,i]$$

$$\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }\left\{\sum _{j=1}^{n}T[i,j]C[i,{\pi }_j],\sum _{j=1}^{n}T[j,{\pi }_i^{-1}]C[j,i]\right\}$$

$$\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }\left\{\sum _{j=1}^{n}T[i,j]C[i,{\pi }_j]+\sum _{j=1}^{n}T[j,{\pi }_i^{-1}]C[j,i]\right\}$$

$$\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\sum _{i=1}^n\sum _{j=1}^n\sum _{k=1}^n(Acc[i,s_j^{\ast }]-Acc[i,s_{j-1}^{\ast }])C[i,k]\pi [j,k]$$

$$\text{centering}\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\sum _{j=1}^n\sum _{k=1}^n(F[s_j^{\ast },k]-F[s_{j-1}^{\ast },k])\pi [j,k]\text{@add@centering}$$

$$\text{centering}\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\sum _{j=1}^n\sum _{k=1}^{n}Cost[s_j^{\ast },s_{j-1}^{\ast },k]\pi [j,k]\text{@add@centering}$$

$$\text{centering}\underset{s_1^{\ast },\cdots s_{p-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\sum _{j=1}^{n}Cost[s_j^{\ast },s_{j-1}^{\ast },{\pi }_j]\text{@add@centering}$$

$$\sum _{j=1}^n\omega (v_{j,i_j},v_{j-1,i_{j-1}},{\pi }_j)$$

$$\sum _{k=1}^{i-1}\omega (v_{k-1,j_{k-1}},v_{k,j_k},{\pi }_k)\le Cost(v_{i-1,j_{i-1}},\pi \setminus \{l\})$$

$$\sum _{k=1}^i\omega (v_{k-1,j_{k-1}},v_{k,j_k},{\pi }_k)\le Cost(v_{i,j},\pi )$$

$$\underset{j\in [n]}{\max }\omega (v_{j,i_j},v_{j-1,i_{j-1}},{\pi }_j)$$

$$\underset{s_1^{\ast },\cdots s_{n-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }$$

$$\underset{s_1^{\ast },\cdots s_{p-1}^{\ast }\in S}{\min }\underset{\pi \in \Pi (n)}{\min }\underset{i\in [n]}{\max }\left\{\begin{array}{r}\sum _{j=1}^p\sum _{k=1}^{p}V[s_j^{\ast },s_{j-1}^{\ast }][i]C[i,k]\pi [j,k]\\ \sum _{j=1}^p\sum _{k=1}^{p}V[s_k^{\ast },s_{k-1}^{\ast }][j]C[j,i]\pi [k,i]\end{array}\right\}$$

$$\underset{i\in [n]}{\max }\left\{\sum _{j=1}^{n}T[i,j]C[i,{\pi }_j],\sum _{j=1}^{n}T[j,{\pi }_i^{-1}]C[j,i]\right\}$$

$$\omega (v_{i,j},v_{i^{\prime },j^{\prime }})=\{\begin{array}{rl}(s_{j^{\prime }},s_{j^{\prime }},0,\cdots ,0),& if{i}^{\prime }\in [1,k]\\ (0,0,0,\cdots ,0),& if{i}^{\prime }\in [k+1,2k+2]\end{array}$$

$$T[l,j]=\{\begin{array}{rl}{s}_{i_j},& ifl=1or2,j\in [1,k]\\ 0,& otherwise\end{array}$$

Full text

1 Introduction

The Massively Parallel Computation model [33], MPC for short, has been a well acknowledged model to study parallel algorithms [20, 2, 7, 9, 10, 25, 39, 59] ever since it was proposed. Compared to other parallel computation models such as PRAM [35], BSP [60], LogP [19] and so on, the advantage of the MPC model lies in its simplicity and the power to capture the essence of computation procedure of modern share-nothing clusters. In the MPC model, computation proceeds in synchronous rounds, where in each round the computation machines first communicate with each other, then conduct local computation. Any pair of machines can communicate in a point-to-point manner, and all the communication messages can be transferred without congestion.

Although the MPC model is simple and powerful, one of its most important shortcomings is revealed by some recent works [11, 32], which is the strong assumption of homogeneity. All the machines in MPC model are considered as identical, and the communication bandwidth between any pair of machines are identical too [33]. In realistic parallel environment, the assumption of identical computation machines can be satisfied in most cases, but the assumption of identical communication bandwidth can not. Typically, a cluster consists of several racks connected by slower communication channels, and each rack includes several machines connected by faster communication channels. Thus, the communication bandwidth of in-rack and across-rack communication differ significantly, which refutes the basic assumption of homogeneous communication network in MPC model.

In order to tackle this shortcoming of the MPC model, a new topology aware massively parallel computation model was proposed and studied in [11, 32]. The computation machines are still identical in this model111There may be non-computational machines in this model, though., but the communication bandwidth between different pair of machines are different. This model was first proposed in recent works [11], where the underlying communication network is represented as a graph, and the edges are assigned with a weight which represents the communication bandwidth. However, the paper [11] only declared the new model but did not give any theoretical results. The other work [32] considered three data processing tasks on this model, which are set intersection, Cartesian product and sorting. Algorithms and lower bounds about the communication cost optimization problems for the three tasks were proposed [32]. However, the authors of [32] restricted the underlying communication network to trees, and the algorithm and lower bounds given in that paper can not be generalized to graphs other than trees.

In this paper, we follow the line of research started by [11, 32], and consider the topology aware massively parallel computation model in a more general case, where the underlying communication network is a complete weighted graph. In this sense, our work is a complement to the work in [32]. The goal of this paper is also to minimize the communication cost. However, unlike the work in [32] which considers specific computation tasks, in this paper we define general communication cost minimization problems that capture the characteristics of a variety of computation tasks.

1.1 Description of the research problems in this paper

1.1.1 The WMPC Model

We first give a more detailed description of the computational model considered in this paper, which is called Weighted Massively Parallel Computation (WMPC) model.

In WMPC model, there are $n$ computation machines with identical computational power. The communication network is modeled as a weighted complete graph, which is represented by a $n\times n$ matrix $C$. $C$ is called the communication cost matrix from now on, and it is considered as a known parameter of the WMPC model. $C[i,j]$ is the communication cost between computation machine $i$ and $j$ for $1\leq i,j\leq n$, where larger value implies larger communication cost or communication latency. $C[i,i]$ is set to 0 for $1\leq i\leq n$. It is assumed that all pairs of machines can communicate in a point-to-point way which is in accordance with the original MPC model, and thus $C[i,j]<\infty$ holds for $1\leq i,j\leq n$. The matrix $C$ is not necessary to be symmetric, i.e., $C[i,j]$ may not be equal to $C[j,i]$.

The computation on WMPC proceeds in synchronous rounds which behaves the same with the original MPC model. In each round, the computation machines first communicate with each other, then conduct local computation.

The initial data distribution plays an important part in the problems studied in this paper. A lot of former research works on MPC model assume that the data are uniformly split across the machines [25, 9]. In this paper, it is assumed that the data can be arbitrarily distributed, and the amount of data placed at each machine is known in advance. This is also the same assumption adopted in [10, 32].

1.1.2 Objective functions

The goal of this paper is to minimize the communication cost under WMPC model, which is divided into send cost and receive cost. If $\alpha$ amount of data is transferred from machine $i$ to machine $j$, it incurs $\alpha\cdot C[i,j]$ send cost to machine $i$, and $\alpha\cdot C[i,j]$ receive cost to machine $j$. Denote $send_{i}$ and $rcv_{i}$ to be the send and receive cost of machine $i$ for $1\leq i\leq n$, then we define the following four objective functions.

Total cost (TOTAL): $\sum_{i=1}^{n}send_{i}$.

Bottleneck cost (BTNK): $\max_{i=1}^{n}rcv_{i}$.

Maximum of send and receive cost (MSR): $\max_{i=1}^{n}\{send_{i},rcv_{i}\}$.

Sum of send and receive cost (SSR): $\max_{i=1}^{n}\{send_{i}+rcv_{i}\}$.

For the TOTAL cost function, it holds that $\sum_{i=1}^{n}send_{i}=\sum_{i=1}^{n}rcv_{i}$. For the BTNK cost function, we choose to use the bottleneck of the receive cost rather than send cost, since the receive cost also reflects the workload of local computation. The MSR cost function is used in [57] on the classical MPC model, and in this paper we investigate its properties on WMPC model. The SSR cost function is closely related to MSR cost function, and is basically a 2-approximation of MSR cost.

Note that the send and receive cost is defined based on the amount of data transferred between two machines. For different commutation task and different communication pattern, the way of calculating the amount of transferred data will be different. Next we will use concrete computation tasks such as parallel sorting and join as the introducing example, analyze their communication patterns, and define the problems to be studied in this paper. We will introduce two problems, named Data Redistribution Problem and Data Allocation Problem.

1.1.3 The Data Redistribution Problem

Consider the following parallel sorting algorithm on classical MPC model, which is often referred as TeraSort [51]. The algorithm first selects $n-1$ splitters $s_{1}\leq s_{2}\leq\cdots\leq s_{n-1}$ and broadcast the splitters to all machines. The $n-1$ splitters form $n$ intervals $I_{i}=(s_{i-1},s_{i}]$ where $s_{0}=-\infty$ and $s_{n}=\infty$. After obtaining the splitters, each machine sends the local data falling in the $i$-th interval to the $i$-th machine. In such way the data is ordered across the machines. Note that the label of the machines are fixed before the algorithm starts. Then the machines conduct local sorting, and the sorting task can be finished.

Now consider running the parallel sorting algorithm on the WMPC model, and assume that the splitters have been determined. The algorithm described above asks the data in the $i$-th interval to be sent to the $i$-th machine. However, this operation may lead to non-optimal communication cost. Consider the following extreme case. The data are initially inversely sorted across the machines, i.e., for machine $i<j$, the data in machine $i$ are always no less than the data in machine $j$. In such a case, if the $i$-th interval is assigned to the $(n-i)$-th machine, there would be no need to conduct communication. However, if the algorithm asks to send the data in the $i$-th interval to the $i$-th machine, all the data will be totally redistributed, incurring large amount of communication.

Actually, there exist two shortcomings for the above TeraSort algorithm on classical MPC model. First, it neglects the initial data distribution, and neglects the importance of the way to assign the intervals to the machines to minimize the communication cost. Second, it does not consider the difference of communication costs between different pair of machines. By tackling these two points together, the first research problem to be studied in this paper is formed, which is called the Data Redistribution Problem (DRP).

The input of DRP is two $n\times n$ matrices $T$ and $C$. $T[i,j]$ represents the amount of data in the $i$-th machine that fall in the $j$-th interval. The $C$ matrix is the communication cost matrix of the WMPC model. The output is to assign the intervals to the machines, such that the communication cost is minimized. By applying the four communication cost functions introduced in Section 1.1.2, we get four problems denoted as DRP-TOTAL, DRP-BTNK, DRP-MSR and DRP-SSR, respectively. The four problems are studied in Section 2.

1.1.4 The Data Allocation Problem

In the above case of parallel sorting, it is assumed that the splitters are known in advance. However, how to select the splitters to minimize the communication cost is also an important research problem [59], and even a new problem under the WMPC model. For a formal description, let $N$ be the total number of data records to be sorted, and $n$ be the number of machines. Under the assumption that the initial data distribution is known in advance, let $S_{i}=\{s_{i,1},s_{i,2},\cdots,s_{i,l_{i}}\}$, $1\leq i\leq n$, which is the data initially residing in machine $i$. $l_{i}$ is the number of data records in machine $i$, and $\sum_{i=1}^{n}l_{i}=N$. If the splitters are chosen as $s_{1},s_{2},\cdots,s_{n-1}$, they will form $n$ intervals $(s_{j-1},s_{j}]$, where $s_{0}=-\infty$ and $s_{n}=\infty$. Let $T[i,j]=|S_{i}\cap(s_{j-1},s_{j}]|$, which is the number of data records in machine $i$ that falls into the $j$-th interval $(s_{j-1},s_{j}]$. To minimize the communication cost, the problem is to select $n-1$ splitters $s_{1}\leq s_{2}\leq\cdots\leq s_{n-1}$ which split the data into $n$ intervals, then find an assignment from the intervals to the machines, such that the communication cost is minimized. This problem is called Data Allocation Problem (DAP).

Remark. Although DRP and DAP are introduced based on sorting, they can be defined using the idea of virtual machines and physical machines. For DRP, the input $T[i,j]$ can be considered as the amount of data initially residing in physical machine $i$ to be processed by virtual machine $j$, and the output is a permutation which assigns virtual machines to physical machines so that the communication cost is minimized. For DAP, choosing the splitters can be regarded as deciding the data distribution across the virtual machines, since each virtual machine is responsible to collect the data in one interval. In such a point of view, DRP and DAP can be applied to a wide range of concrete problems. Also, DRP and DAP reflect only the problems that can be solved using one synchronous round. It will the future work to study multi-round algorithms on WMPC model.

1.2 Summary of results and paper organization

Summarizing the above descriptions, we have two kinds of problems including DRP and DAP. We also have four kinds of communication cost functions including TOTAL, BTNK, MSR and SSR. 8 problems are obtained by combining two kinds of problems with four kinds of cost functions. The hardness for the 8 problems make up the content of this paper. Table 1 summarizes all the proposed results. Some less important results and some detailed proofs are moved to appendix due to space limitation.

In the rest of this paper, we first introduce some denotations that will be used throughout this paper in Section 1.3, then present the results in Section 2 and 3 following the order given in Table 1. The related work and future work are delayed to Section 4 and 5. Section 6 concludes this paper.

1.3 Denotations

A $m\times n$ matrix $A$ is denoted as $A^{m\times n}$. The element in $A$ at row $i$ and column $j$ is denoted as $A[i,j]$. The set of consecutive integers $\{i,i+1,i+2,\cdots,j\}$ is denoted as $[i,j]$. The set of integers $\{1,2,\cdots,n\}$ is denoted as $[n]$.

A permutation on $[n]$ is a one-to-one mapping from $[n]$ to $[n]$, and it is usually denoted as $\pi$. The set of all permutations on $[n]$ is denoted as $\Pi(n)$. Denote $\pi_{i}$ as the image of $i$ under $\pi$. If $\pi_{i}=j$, it is also said that $i$ is assigned to $j$ by permutation $\pi$. We also use $\pi^{-1}$ to denote the inverse permutation of $\pi$, i.e., if $\pi_{i}=j$ then $\pi^{-1}_{j}=i$.

2 The Data Redistribution Problem Series

Definition 1 (DRP)

Input: A $n\times n$ transmission matrix $T^{n\times n}$ and a $n\times n$ communication cost matrix $C^{n\times n}$, where $C[i,i]=0$ for $i\in[n]$.

Output: find a permutation $\pi\in\Pi(n)$ such that the communication cost function chosen from TOTAL, BTNK, MSR and SSR is minimized. Formally,

DRP-TOTAL:

[TABLE]

DRP-BTNK:

[TABLE]

DRP-MSR:

[TABLE]

DRP-SSR:

[TABLE]

2.1 DRP-TOTAL

Theorem 2.1

DRP-TOTAL is equivalent to the Linear Assignment Problem (LAP) [5].

Proof

The 0-1 integral linear programming (LP) formation of DRP-TOTAL is

[TABLE]

We have

[TABLE]

Now define another matrix $F^{n\times n}$ as $F[j,k]=\sum_{i=1}^{n}T[i,j]C[i,k]$, $j,k\in[n]$, and the objective function in Equation 1 is transformed into $\min\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}{F[i,j]x_{ij}}$, which is equivalent to the linear programming formation of Linear Assignment Problem. ∎

Corollary 1

DRP-TOTAL can be solved in $O(n^{3})$ time.

Proof

Transforming DRP-TOTAL to LAP needs $\Theta(n^{3})$ time. The Hungarian algorithm [48] to solve LAP needs $O(n^{3})$ time. Then the corollary follows. ∎

2.2 DRP-BTNK

Theorem 2.2

DRP-BTNK is equivalent to the Linear Bottleneck Asssignment Problem (LBAP) [13].

Proof

The 0-1 integral LP formation of DRP-BTNK is

[TABLE]

Define another matrix $F^{n\times n}$ as $F[k,i]=\sum\limits_{j=1}^{n}T[j,k]C[j,i]$, and the objective function in Equation 2 becomes $\min\max\limits_{i\in[n]}\sum\limits_{k=1}^{n}F[k,i]x_{ki}$, which is equivalent to the LP formation of LBAP. ∎

Corollary 2

DRP-BTNK can be solved in $O(n^{3})$ time.

Proof

Transforming DRP-BTNK to LBAP needs $\Theta(n^{3})$ time. The algorithm to solve LBAP needs $O(n^{3})$ time [54]. Then the corollary follows. ∎

2.3 DRP-MSR

Theorem 2.3

DRP-MSR is NP-complete.

Proof

We reduce the following PARTITION problem to DRP-MSR, which is well known to be NP-complete.

Input: a set $S$ of $n$ integers $S=\{s_{1},s_{2},\cdots,s_{n}\}$, where $\sum\limits_{s_{i}\in S}s_{i}=B$.

Output: decide whether there exists a partition $(S_{1},S_{2})$ of $S$ s.t. $S_{1}\cap S_{2}=\emptyset,S_{1}\cup S_{2}=S$, and $\sum\limits_{s_{i}\in S_{1}}s_{i}=\sum\limits_{s_{i}\in S_{2}}s_{i}=B/2$. Such partition is called perfect.

For any instance of PARTITION, construct an instance of DRP-MSR. The two matrices $T$ and $C$ as the input of DRP-MSR are of size $(2n+2)\times(2n+2)$. Let

[TABLE]

Let $\Delta$ be a sufficiently large integer (it suffices to set $\Delta=B$), and

[TABLE]

Since the communication cost between machine 1 and 2 are set to $\Delta$ which is sufficiently large, and $T[1,i]\neq 0$, $T[2,i]\neq 0$ for $i\in[1,n]$, then if some $i\in[1,n]$ is assigned to machine 1 or 2 by permutation $\pi$, e.g., $\pi_{i}=1$, there will be $T[1,i]C[1,\pi_{i}]=T[1,i]\cdot\Delta$, which incurs a sufficiently large communication cost between machine 1 and 2. Thus, the integers in $[1,n]$ must be assigned to some $j\in[3,2n+2]$. Notice that machine 1 connects only to machines $[3,n+2]$, and machine 2 connects only to machines $[n+3,2n+2]$, with communication cost set to 1. Then if some $i\in[1,n]$ is assigned to some $j\in[3,n+2]$, it incurs $a_{i}$ send cost to machine 1, and 0 send cost to machine 2. On the other hand, if some $i\in[1,n]$ is assigned to some $j\in[n+3,2n+2]$, it incurs 0 send cost to machine 1, and $a_{i}$ send cost to machine 2. This indeed reflects the cost of the partition.

Actually, for any permutation $\pi\in\Pi(2n+2)$ where $\pi_{i}\neq 1,2$ for $i\in[1,n]$, the send cost of machine 1 under $\pi$ is the sum of the elements in the set $\{s_{i}\mid\pi_{i}\in[3,n+2]\}$, and the send cost of machine 2 is the sum of the elements in the set $\{s_{i}\mid\pi_{i}\in[n+3,2n+2]\}$. It can be easily verified that the receive cost of other machines are relatively small, since each of the machines in $[3,2n+2]$ only receives one element. Furthermore, the receive cost of machine 1 and 2, and the send cost of the other machines are all 0. Denote $S_{\pi,1}=\{s_{i}\mid\pi_{i}\in[3,n+2]\},S_{\pi,2}=\{s_{i}\mid\pi_{i}\in[n+3,2n+2]\}$, and $(S_{\pi,1},S_{\pi,2})$ forms a partition of $S$. In such way, the permutation $\pi$ corresponds to the partition $(S_{\pi,1},S_{\pi,2})$. Finally, the MSR cost of the permutation $\pi$, denoted as $MSR_{\pi}$, is obtained by taking maximum over the send cost of machine 1 and 2, i.e., $MSR_{\pi}=\max\left\{\sum\limits_{s_{i}\in S_{\pi,1}}s_{i},\sum\limits_{s_{i}\in S_{\pi,2}}s_{i}\right\}$.

By the above discussion, we can claim that above construction is indeed a reduction from PARTITION to DRP-MSR. If the PARTITION problem admits a perfect partition, then the optimum value of the constructed DRP-MSR instance is exactly $B/2$. If there is no perfect partition for the PARTITION problem, then the optimal value of the constructed DRP-MSR instance must be greater than $B/2$. Then the NP-completeness is proved by observing DRP-MSR is truly in NP. ∎

2.4 DRP-SSR

Theorem 2.4

DRP-SSR is NP-complete.

Proof

The NP-completeness proof for DRP-SSR can be essentially the same with DRP-MSR. In the DRP-MSR instance constructed in the proof of Theorem 2.3, for any machine $i$ it holds that if $send_{i}\neq 0$ then $rcv_{i}=0$, and if $rcv_{i}\neq 0$ then $send_{i}=0$. Thus, the MSR and SSR cost value is the same for that instance. An alternate proof is to reduce 3-PARTITION to DRP-SSR. The 3-PARTITION problem is defined as follows.

Input: $3k$ positive integer numbers $s_{1},\cdots,s_{3k}$ such that $\sum\limits_{j=1}^{3k}s_{j}=kB$, where $B$ is a positive integer.

Output: determine whether there exists a partition $S_{1},\cdots,S_{k}$ of the $3k$ numbers, such that $|S_{l}|=3$ and $\sum\limits_{s_{j}\in S_{l}}{s_{j}}=B$ for all $l\in[k]$.

3-PARTITION is known to be strongly NP-complete [24], i.e., there is no algorithm whose complexity is bounded by a polynomial of $k$ and $B$ that can solve it, unless P=NP.

For any instance of 3-PARTITION, construct an instance of DRP-SSR with $n=3k$ as follows.

Let $T[i,j]=s_{j}$ for $i\in[n],j\in[3k]$. Let $T[i,j]=0$ for $i\in[2k+1,3k],j\in[3k]$. By doing so we have $2k$ machines that all have all the data, and $k$ machines that have no data.

Let $C[i,j]$ be as follows.

[TABLE]

By doing so the $3k$ machines are grouped into $k$ groups each with 3 machines. The 3 machines in each group is connected into a triangle.

For any partition of the $3k$ numbers into $k$ subsets $S_{i}=\{s_{j_{1}},s_{j_{2}},s_{j_{3}}\}$, $i\in[k]$, assign the three numbers $\{s_{j_{1}},s_{j_{2}},s_{j_{3}}\}$ to the three machines $i,k+i,2k+i$ which are in the $i$-th group. Assuming $s_{j_{1}}\geq s_{j_{2}}\geq s_{j_{3}}$, the assignment must assure that $s_{j_{3}}$ is assigned to machine $2k+i$ which initially have no data. $s_{j_{1}}$ and $s_{j_{2}}$ can be assigned to machine $i,k+i$ arbitrarily. Assume w.l.o.g. that $s_{j_{1}}$ is assigned to $i$ and $s_{j_{2}}$ is assigned to $k+i$. It can be verified that this assignment can achieve the smallest possible value of the SSR objective function.

Under this assignment, denote $send_{i}$ and $rcv_{i}$ to be the send and receive cost of machine $i$. For $1\leq i\leq k$, we get

[TABLE]

Since $s_{j_{1}}\geq s_{j_{2}}\geq s_{j_{3}}$, we have $2s_{j_{3}}\leq s_{j_{1}}+s_{j_{2}}+s_{j_{3}}$. Taking maximum over $i,k+i,2k+i$, we get the maximum value of $send+rcv$ cost in the $i$-th group as $s_{j_{1}}+s_{j_{2}}+s_{j_{3}}$. The final cost of DRP-SSR under this assignment is obtained by taking maximum over $i\in[k]$.

If there exists a solution of 3-PARTITION, then the above constructed instance of DRP-SSR has an optimum cost of $B$, since for each subset we have $s_{j_{1}}+s_{j_{2}}+s_{j_{3}}=B$.

If there is no solution of 3-PARTITION, then for the above constructed instance of DRP-SSR, the objective value of SSR cost of any assignment must be greater than $B$. The reason is that there must exist some $i$ such that $send_{i}+rcv_{i}=s_{j_{1}}+s_{j_{2}}+s_{j_{3}}>B$.

The description of the reduction from 3-PARTITION to DRP-SSR is completed, and the NP-completeness of DRP-SSR is proved. ∎

3 The Problem Series of Data Allocation Problem

In this section we study the parameterized hardness and algorithms for the DAP-Cont problem series, parameterized by the number of machines. We will use $N$ to denote the size of the input, and $n$ to denote the number of machines.

Definition 2 (DAP-Cont)

Input: a set $S$ of $N$ integers divided into $n$ subsets $S_{1}=\{s_{1,1},s_{1,2},\cdots,s_{1,l_{1}}\},\cdots,S_{n}=\{s_{n,1},s_{n,2},\cdots,s_{n,l_{n}}\}$, where $n>1$ is the number of machines, and $l_{i}$ is the size of $S_{i}$ satisfying $\sum\limits_{i=1}^{n}l_{i}=N$.

Output: find $n-1$ integers $s^{*}_{1},\cdots s^{*}_{n-1}\in S$ and a permutation $\pi\in\Pi(n)$, such that the communication cost function chosen from TOTAL, BTNK, MSR and SSR is minimized. Formally,

DAP-TOTAL:

[TABLE]

DAP-BTNK:

[TABLE]

DAP-MSR:

[TABLE]

DAP-SSR:

[TABLE]

where $T[i,j]=|S_{i}\cap(s^{*}_{j-1},s^{*}_{j}]|$ and $s^{*}_{0}=-\infty,s^{*}_{n}=\infty$.

3.1 The splitter-graph

We introduce the splitter-graph, which transforms the problem of choosing splitters to choosing a path in a special graph. Given a set $S=\{s_{1},s_{2},\cdots,s_{N}\}$ of integers, assuming $s_{1}\leq s_{2}\leq\cdots\leq s_{N}$, and a parameter $n$, construct a graph $G(V,E)$ as follows. For $i\in[n-1],j\in[N]$, construct a vertex $v_{i,j}$. Let $v_{0,0}$ be the starting vertex $-\infty$, and $v_{n,N+1}$ be the end vertex $\infty$. Let $(v_{i,j},v_{i^{\prime},j^{\prime}})\in E$ iff $i+1=i^{\prime}$ and $j<j^{\prime}$. In such way, a vertex $v_{i,j}$ represents a splitter $s_{j}$ placed in the $i$-th position, and a path $(-\infty,v_{1,i_{1}},v_{2,i_{2}}\cdots v_{n-1,i_{n-1}},\infty)$ represents selecting $s_{i_{1}},s_{i_{2}},\cdots s_{i_{n-1}}$ as splitters. From now on, a splitter-graph based on set $S$ with parameter $n$ will be denoted as $G_{s}(V,E,S,n)$. The discussions in the rest of this section will depend on the splitter-graph with different definition of edge weights.

3.2 FPT algorithm of DAP-TOTAL and DAP-BTNK

The FPT algorithm of DAP-TOTAL and DAP-BTNK is based on the following transformation. Note that the transformation can be done in polynomial time. Given an instance of DAP-TOTAL, denote $S=\{s_{1},s_{2},\cdots,s_{N}\}$ and assume $s_{1}\leq s_{2}\leq\cdots\leq s_{N}$. Let $s_{0}=-\infty$ and $s_{N+1}=\infty$. Let $Acc[i,j]=|S_{i}\cap(-\infty,s_{j}]|,i\in[n],j\in[0,N+1]$. Slightly abusing denotation, let $\pi^{n\times n}$ be a matrix defined based on the permutation $\pi$, such that $\pi[i,j]=1$ if $\pi_{i}=j$, and $\pi[i,j]=0$ otherwise, $i,j\in[n]$.

Under the above denotations, DAP-TOTAL can be transformed into the following form.

[TABLE]

where $s_{0}^{*}=-\infty$ and $s_{p}^{*}=\infty$. Let $F[j,k]=\sum\limits_{i=1}^{p}Acc[i,j]C[i,k]$, $j\in[0,N+1],k\in[n]$, then the above equation is transformed into

[TABLE]

Let $Cost[i,j,k]=F[i,k]-F[j,k]$, $0\leq j<i\leq N+1,k\in[n]$, then Equation 4 is transformed into

[TABLE]

If $\pi$ is represented by a permutation, we get

[TABLE]

Now we can associate the above $Cost$ function to the spiltter-graph. For each edge $(v_{i,j},v_{i^{\prime},j^{\prime}})$ in the splitter-graph and each $l\in[n]$, let $\omega(v_{i,j},v_{i^{\prime},j^{\prime}},l)=Cost[j^{\prime},j,l]$. The function $\omega$ can be regarded as assigning $p$ weights to each edge, and each weight is associated with a label $l\in[n]$. Now, we have the following splitter-graph formation of DAP-TOTAL.

Definition 3

Input: a splitter-graph $G_{s}(V,E,S,n)$, the weight function $\omega:V\times V\times[n]\rightarrow\mathcal{R}$ of DAP-TOTAL.

Output: a path $(-\infty,v_{1,i_{1}},v_{2,i_{2}}\cdots v_{n-1,i_{n-1}},\infty)$, and a permutation $\pi$, such that the following cost is minimized

[TABLE]

According to the above transformation, Definition 3 is equivalent to the original definition of DAP-TOTAL.

3.2.1 FPT Algorithm for Decision-DAP-TOTAL

We prove the following decision version of DAP-TOTAL is FPT.

Definition 4 (Decision-DAP-TOTAL)

Input: a splitter-graph $G_{s}(V,E,S,n)$, the weight function $\omega:V\times V\times[n]\rightarrow\mathcal{R}$ of DAP-TOTAL, a threshold value $\alpha$, and parameter $n$.

Output: Is the optimum value of DAP-TOTAL less than $\alpha$?

We need the following definition of partial permutations. A partial permutation $\pi$ is a function defined on $[i]$ where $i\in[n]$, such that $\pi_{j},\pi_{k}\in[n]$ and $\pi_{j}\neq\pi_{k}$ for $1\leq j\neq k\leq i$. Here $\pi_{j}$ is the image of $j$ under $\pi$. Given a partial permutation $\pi$ whose definition domain is $[i]$, and an integer $l\in[n]$, let $l\in\pi$ denote that there exists some $j\in[i]$ such that $\pi_{j}=l$. Given an integer $l\notin\pi$, let $\pi\cup\{l\}$ be a new partial permutation $\pi^{\prime}$ defined on $[i+1]$ such that $\pi^{\prime}_{i+1}=l$ and $\pi^{\prime}_{j}=\pi_{j}$ for $j\in[i]$. Given an integer $l=\pi_{i}$, let $\pi\setminus\{l\}$ be a partial permutation $\pi^{\prime}$ defined on $[i-1]$, such that $\pi^{\prime}_{j}=\pi_{j}$ for all $j\in[i-1]$. Denote $\Phi$ as the empty partial permutation.

Algorithm 1 is the FPT algorithm for Decision-DAP-TOTAL. The algorithm maintains two arrays of length $O(n!)$ for each vertex $v_{i,j}$, namely Perm($v_{i,j}$) and Cost($v_{i,j},\pi$). Perm ($v_{i,j}$) stores all the feasible partial permutations for the path from $-\infty$ to $v_{i,j}$, and Cost($v_{i,j},\pi$) stores the partial accumulated cost value corresponding to the partial permutation $\pi$.

Theorem 3.1

*At the end of the $i$-th iteration of the outer-most $for$-loop in Algorithm 1, $i\in[n]$, it holds that

(1) each $\pi\in Perm(v_{i,j})$ is a partial permutation, and

(2) $\pi\in Perm(v_{i,j})$ iff there exists a path $(-\infty,v_{1,j_{1}},\cdots,v_{i-1,j_{i-1}},v_{i,j_{i}})$ such that $\sum\limits_{k=1}^{i}\omega(v_{k-1,j_{k-1}},v_{k,j_{k}},\pi(k))\leq Cost(v_{i,j},\pi)$.*

Proof

(1) Straightforward. The condition in Line 1 of Algorithm 1 ensures that if $l\in\pi$ then $\pi\cup\{l\}$ will not be added into $Perm(v_{i,j})$, which ensures that each $\pi\in Perm(v_{i,j})$ is a partial permutation.

(2) The proof proceeds by induction on $i$. As start point of induction where $i=0$, the path is from $-\infty$ to $-\infty$, i.e., a single vertex. Then the claim trivially holds.

Suppose the claim holds at the end of the $(i-1)$-th iteration, and consider the $i$-th iteration. According to Line 9 and 10 in Algorithm 1, $\pi\in Perm(v_{i,j})$ if and only if there is an edge $(v_{i-1,j_{i-1}},v_{i,j})$ and an integer $l=\pi_{i}$ such that $Cost(v_{i-1,j_{i-1}},\pi\setminus\{l\})+\omega(v_{i-1,j_{i-1}},v_{i,j},l)\leq\alpha$. By induction hypothesis, the partial permutation $\pi\setminus\{l\}\in Perm(v_{i-1,j_{i-1}})$ if and only if there exists a path from $-\infty$ to $v_{i-1,j_{i-1}}$, $(-\infty,v_{1,j_{1}},\cdots,v_{i-1,j_{i-1}})$, such that

[TABLE]

Now adding the edge $(v_{i-1,k},v_{i,j})$ to the path, and adding $l$ to the partial assignment $\pi\setminus\{l\}$, we obtain a path from $-\infty$ to $v_{i,j}$ and a partial assignment $\pi$ such that

[TABLE]

By induction, the claim is proved. ∎

According to Theorem 3.1, by applying the induction to the vertex $\infty$, it holds that $Perm(\infty)\neq\emptyset$ if and only if there exists a path $(-\infty,v_{1,j_{1}},\cdots,v_{n-1,j_{n-1}},\infty)$, such that $\sum\limits_{k=1}^{n}\omega(v_{k-1,j_{k-1}},v_{k,j_{k}},\pi_{k})\leq Cost(\infty,\pi)\leq\alpha$, where $\pi$ is a permutation in $Perm(\infty)$. By the splitter-graph formation of Decision-DAP-TOTAL (Definition 3), it is equivalent to that the optimum value of DAP-TOTAL is less than $\alpha$. This completes the correctness proof of Algorithm 1.

Theorem 3.2

Decision-DAP-TOTAL is FPT.

Proof

This theorem is true since Algorithm 1 solves Decision-DAP-TOTAL in $O(N^{2}n!n)$ time. Note that here the number of machines $n$ is regarded as the parameter, and this complexity is polynomial in the input size $N$. ∎

3.2.2 FPT Algorithm for DAP-BTNK

Using a transformation similar with that for DAP-TOTAL, we have the following splitter-graph formation for DAP-BTNK.

Definition 5

Input: a splitter-graph $G_{s}(V,E,S,n)$, the weight function $\omega:V\times V\times[n]\rightarrow\mathcal{R}$ of DAP-BTNK.

Output: a path $(-\infty,v_{1,i_{1}},v_{2,i_{2}}\cdots v_{n-1,i_{n-1}},\infty)$, and a permutation $\pi$, such that the following cost is minimized

[TABLE]

The decision version of DAP-BTNK has an extra value $\alpha$ as input, and asks whether the optimum value of DAP-BTNK is less than $\alpha$. We first propose the FPT algorithm for the decision version, which is given as Algorithm 2. It needs one array for each vertex $v_{i,j}$ which is Perm($v_{i,j}$). The algorithm is similar with that for Decision-DAP-TOTAL, only changing the sum-check (Line 1 in Algorithm 1) to maximum check (Line 2 in Algorithm 2). Thus the correctness proof of this algorithm is similar with Theorem 3.1, and it is omitted.

Theorem 3.3

DAP-BTNK is FPT.

Proof

We can use the algorithm for Decision-DAP-BTNK to solve DAP-BTNK. The idea is similar with the two-phase algorithm for Linear Bottleneck Assignment Problem given in Section 2.4. In the first phase the algorithm chooses some possible value from the input weight function $\omega:V\times V\times[n]\rightarrow\mathcal{R}$. In the second phase Algorithm 2 is invoked by setting $\alpha$ as the selected weight value. Since the number of possible values of the input weight function $\omega:V\times V\times[n]\rightarrow\mathcal{R}$ is at most $N^{2}n$, then Algorithm 2 is invoked for at most $N^{2}n$ times. Thus, the two-phase algorithm for DAP-BTNK is still FPT. ∎

3.3 W[1]-completeness of DAP-MSR and DAP-SSR

We first prove that the two problems are in W[1].

Theorem 3.4

DAP-MSR and DAP-SSR are in W[1].

Proof

The proof is based on the machine characterization of the W[1] class proposed in [16]. The following definition and theorem must be cited from [16] to support this proof.

Definition 6 (W-program, [16])

A nondeterministic RAM program $\mathbb{P}$ is a W-program, if there is a function $f$ and a polynomial $p$ such that for every input $(x,k)$ with $|x|=n$, the program $\mathbb{P}$ on every run

(1) performs at most $f(k)\cdot p(n)$ steps;

(2) at most $f(k)$ steps are nondeterministic;

(3) at most the first $f(k)\cdot p(n)$ registers are used;

(4) at every point of the computation the registers contain numbers $\leq f(k)p(n)$;

Theorem 3.5 ([16])

Let $Q$ be a parameterized problem. Then $Q\in$ W[1] if and only if, there is a computable function $h$ and a W-program $\mathbb{P}$ deciding $Q$ such that for every run of $\mathbb{P}$ all nondeterministic steps are among the last $h(k)$ steps of the computation, where $k$ is the parameter.

First we should note that if term (4) in Definition 6 is to be satisfied, the elements in the communication cost matrix, and the edge weights in the splitter-graph, should be bounded by $f(n)p(N)$. Under this constraint, we describe the W-program for DAP-MSR which is quite simple.

(1) Transform the input to splitter-graph formation.

(2) Non-deterministically guess a path with length $n$. Note that by the definition in [16], the nondeterministic machine can guess an integer in one nondeterministic step, rather than guess a single bit. In such way the nondeterministic machine only need to perform $n$ nondeterministic steps to guess the path.

(3) Enumerate all the $n!$ permutations, and find the optimal permutation with the smallest MSR cost value.

It is obvious that the above program (1) performs at most $O(N^{2}n)+O(n!)$ steps, (2) perform only $n$ nondeterministic guess steps, (3) uses $O(N^{2}n)$ registers to record the input, $O(n)$ registers to record the selected path, $O(n)$ registers to record the enumerated permutation, and constant registers to conduct extra numeric operations. Furthermore, after the path is guessed, the subsequent computation needs $O(n!)$ time which satisfies the condition given in Theorem 3.5. ∎

We then prove the W[1]-hardness of the two problems. With an idea similar with that in Section 3.2, we first transform DAP-MSR and DAP-SSR into a splitter-graph formation. We only describe the transformation for DAP-MSR, and it is similar for the other. Using the same denotations used in Equation 3, we have the following equivalent form for DAP-MSR.

[TABLE]

Let $V$ be a $N\times N$ matrix where each element is a vector of length $n$, and let $V[j,k][i]=Acc[i,j]-Acc[i,k]$, $i\in[n],j,k\in[0,N+1]$, then Equation 7 is transformed into

[TABLE]

Next we give the following splitter-graph formation of DAP-MSR. For each edge $(v_{i,j},v_{i^{\prime},j^{\prime}})$ in the splitter-graph, let $\omega(v_{i,j},v_{i^{\prime},j^{\prime}})=V[j^{\prime},j]$, i.e., each edge is associated with a vector of length $n$. In such way, each path from $-\infty$ to $\infty$ corresponds to $n$ vectors of length $n$, and can form a $n\times n$ matrix $T$. It remains to solve the DRP problem, taking this matrix $T$ and the communication cost matrix $C$ as input. The formal definition is given as follows.

Definition 7

Input: a splitter-graph $G_{s}(V,E,S,n)$, the edge weight function $\omega:V\times V\rightarrow\mathcal{R}^{n}$ of DAP-MSR, the communication cost matrix $C^{n\times n}$.

Output: a path $(-\infty,v_{1,k_{1}},v_{2,k_{2}}\cdots v_{n-1,k_{n-1}},\infty)$, which corresponds a matrix $T^{n\times n}$ where $T[i,j]=\omega(v_{j,k_{j}},v_{j-1,k_{j-1}})[i]$, and a permutation $\pi$, such that the following MSR cost function is minimized:

[TABLE]

To prove the W[1]-hardness of the problem, we introduce an intermediate problem called Selecting-PARTITION. The idea is to reduce $k$-clique, which is W[1]-complete, to Selecting-PARTITION, and reduce Selecting-PARTITION to DAP-MSR (and similarly to DAP-SSR).

Definition 8 (Selecting-PARTITION)

Input: $n$ integers $S=\{s_{1},s_{2},\cdots,s_{n}\}$, target sum value $B$, and parameter $k$.

Output: decide whether there exists a set $A\subset S$ with $|A|=k$, such that $A$ is a Yes-instance of PARTITION, i.e., there exists $A_{1},A_{2}$ such that $A_{1}\cap A_{2}=\emptyset,A_{1}\cup A_{2}=A$ and $\sum_{s_{i}\in A_{1}}s_{i}=\sum_{s_{i}\in A_{2}}s_{i}=B/2$.

Lemma 1

There is a parameterized reduction from $k$-clique to Selecting-PARTITION.

Proof

Given an instance $G=(V,E)$ of $k$-clique, construct an instance of Selecting-PARTITION with parameter $k+k(k-1)/2$ as follows. Assume w.l.o.g. that $V=[n]$, and each vertex in $V$ is labeled by an integer $i\in[n]$. Choose a prime number $q$ with $q>k$. Initially let $S=\emptyset$. For each vertex $i$, add the integer $(k-1)q^{i}$ into S. For each edge $(i,j)\in E$, add $q^{i}+q^{j}$ into S. The construction can be done in $O(|V|+|E|)=O(n^{2})$ time.

For ease of reference let $S_{v}=\{(k-1)q,(k-1)q^{2},\cdots,(k-1)q^{n}$ }, and let $S_{e}$ be the set of integers associated with the edges, where each integer is of the form $q^{i}+q^{j}$. It can be verified that $S_{v}\cap S_{e}=\emptyset$.

If there exists a $k$-clique in $G$ where the vertexes are $i_{1},i_{2},\cdots,i_{k}$, then the vertexes and edges in the clique correspond with two sets of integers $A_{1}=\{(k-1)q^{i_{1}},\cdots,(k-1)q^{i_{k}}\}$ and $A_{2}=\{q^{i_{1}}+q^{i_{2}},\cdots,q^{i_{j-1}}+q^{i_{j}}\}$. It is easy to see that $\sum\limits_{s_{i}\in A_{1}}s_{i}=\sum\limits_{s_{i}\in A_{2}}s_{i}$. The set $A=A_{1}\cup A_{2}$ is of size $k+k(k-1)/2$.

If there exists a set $A\subset S$ where $|A|=k+k(k-1)/2$ that is a Yes-instance of PARTITION, there are two cases. The first case is that there is at least one integer in $A$ that is from $S_{v}$, say, $(k-1)q^{i}$ for some $i$. Then there must be $(k-1)$ integers from $S_{e}$ which correspond to $(k-1)$ edges incident to the vertex $i$. The $(k-1)$ edges then lead to $k-1$ different vertexes, and that is totally $k$ vertexes plus vertex $i$. They must form a $k$-clique, otherwise the corresponding integers will not be a Yes-instance of PARTITION. The second case is that all the $k+k(k-1)/2$ integers are from $S_{e}$. In this case the $k+k(k-1)/2$ integers corresponds to $k+k(k-1)/2$ edges which form a $(k+1)$-clique, and the $(k+1)$-clique must contain a $k$-clique. ∎

Lemma 2

There is a parameterized reduction from Selecting-PARTITION to DAP-MSR and DAP-SSR.

Proof

We only describe the reduction to DAP-MSR, and the proof is similar for DAP-SSR. Given an instance of Selecting-PARTITION, construct a splitter-graph formation of DAP-MSR as follows. First construct a splitter-graph $G_{s}(V,E,S,2k+2)$. For each edge $(v_{i,j},v_{i^{\prime},j^{\prime}})$ in the splitter-graph, let

[TABLE]

where the length of the vectors are $2k+2$.

By such construction, each path $(-\infty,v_{1,i_{1}},v_{2,i_{2}}\cdots v_{2k+1,i_{2k+1}},\infty)$ corresponds to the following matrix $T^{(2k+2)\times(2k+2)}$

[TABLE]

Let the communication cost matrix $C^{(2k+2)\times(2k+2)}$ be as follows.

[TABLE]

Thus, any selected $k$ integers $s_{1},s_{2},\cdots,s_{k}$ from $S$ correspond to two matrices $T^{(2k+2)\times(2k+2)}$ and $C^{(2k+2)\times(2k+2)}$. We can see that the structure of the two matrices $T^{(2k+2)\times(2k+2)}$ and $C^{(2k+2)\times(2k+2)}$ are the same with that used in the proof of NP-completeness of DRP-MSR. By the reduction from PARTITION to DRP-MSR, the $k$ integers $s_{1},s_{2},\cdots,s_{k}$ forms a Yes-instance to PARTITION if and only if the corresponding matrices $T^{(2k+2)\times(2k+2)}$ and $C^{(2k+2)\times(2k+2)}$ form an instance of DRP-MSR whose optimum cost value is $B/2$. We conclude the proof by observing the above reduction takes $O(kN^{2})$ time, and the parameter of the constructed DAP-MSR instance is $n=2k+2$, which satisfies the definition of parameterized reduction. ∎

4 Related Works

4.1 Parallel computation models

This paper is based on the topology-aware MPC model [11, 32], which is almost new. The history of modeling parallel computation is long though, and there exist a lot of models such as PRAM, LogP, CONGEST and so on. Based on our observation, there are three important aspects for parallel computation, which are local computation, communication and data exchange between memory hierarchies. A good model for parallel computation should consider at least one aspect in detail, and neglect the other aspects if necessary. We categorize the existing models into the following three classes according to which aspect of parallel computation that the model emphasize. An important note is that there is no research work that consider all the three aspects in a single model by now.

4.1.1 Models that emphasize communication

Most models consider communication as the most important aspect of parallel computation. Some of these models even totally neglect the local computation, assuming any local computation can be done in a single unit of time. There indeed exist many important contents that is worth considering for communication, such as network topology, synchronization, latency, link capacity, routing, and so on.

A lot of early models are dedicated to model the network topology, such as hypercube network [47] and switch network [37]. See [22] for a survey. PRAM [35] uses a shared memory to exchange message between processors, and the strategy of exclusive/non-exclusive read/write leads to different variants of PRAM, such CREW (concurrent read, exclusive write), EREW, CRCW and so on. The processors are assumed to have arbitrarily strong computational power. PRAM is the most acknowledged model for theoretical researches in parallel computation. See [29] for a survey on PRAM simulation methods. BSP [60] model emphasizes synchronization, where the computation is divided into synchronized supersteps. LogP [19] emphasizes the cost of message exchanging, where $L$ stands for latency, $o$ stands for overhead, and $g$ stands for gap. CONGEST [52] emphasizes the network topology and link capacity, which is mostly used to study graph problems [4, 14, 28]. The communication network in CONGEST has the same topology with the graph problem which it considers, and the length of the messages between any two processors in any round is restricted to $O(\log{n})$ where $n$ is the number of processors. The Congested Clique model is an variant of CONGEST, where the communication network is a complete graph. Many graph problems were studied on Congested Clique model such as MST [27], maximum matching [23], shortest path [31], etc. There are many theoretical research works on CONGEST and Congested Clique model, and please refer to the references in the papers cited here for more related works. The MPC model [33] is abstracted from MapReduce [20], which can also be regarded as a simplified version of BSP. Finally, the topology aware MPC model [11, 32] is based on the MPC model and involves the consideration for network topology, which directly inspires this work.

4.1.2 Models that emphasize local computation

As we have mentioned, many models neglect the local computation and focus on communication, but it does not reduce the importance of local computation in parallel computation. If the communication cost is the only consideration, it may cause the problem of workload imbalance. Actually balancing the workload is an important aspect in the research of parallel computation which is considered in a lot of research papers from 1990s until today [18, 43, 58].

To the extent of our knowledge, the only model that emphasizes the local computation is the MapReduce model in the original form [20]. The MapReduce model defines the local computation as a Map function and a Reduce function. The Map function transfers the data into key-value pairs. The Reduce function conduct the pre-defined computation on the set of key-value pairs with the same key. The communication in MapReduce model is implicit and automatically excecuted by the underlying framework, which gathers all the key-value pairs with the same key to the same machine. Therefore, the main focus of MapReduce model is how to design the Map and Reduce functions, which are both local computation functions.

After the MapReduce model is abstracted into the MPC model [33, 42], the consideration of local load is still a main focus in many research works [59], especially in the study of the join operation in database theory [9, 10, 38, 39, 40].

4.1.3 Models that emphasize memory hierarchy

The memory hierarchy of modern processors includes CPU cache, main memory, external memory and remote storage center. While the idea of memory hierarchy is first established for single processor computation [3, 6], it is of equal importance in parallel computation [44, 55, 62]. The initial motivation of parallel computation is to deal with the problems with size exceeding the power of a single machine. As the amount of data to be processed grows larger and larger, there must be the case that the data size is still too large for each processor even if hundreds of processors are used. Thus, it is necessary to consider the external memory and input/output complexity in parallel computation. However, as far as we know, most recent researches on MPC and Congest model assume that the data can be split fine enough such that the data for each processor can fit in the local memory, which is not the case in realistic environments dealing with massive data. We think there is a lot of research opportunities in this direction.

4.2 Assignment problem

The DRP and DAP are closely related to the assignment problem. As we have shown, DRP-TOTAL is equivalent to Linear Assignment Problem (LAP), DRP-BTNK is equivalent to Linear Bottleneck Assignment Problem (LBAP), and the solution to DAP-BTNK highly relies on the solution to LBAP. We refer the readers for [13] to a good survery of linear assignment problem series.

The Quadratic Assignment Problem (QAP) [45] is harder than LAP which is NP-complete. The permutation formation of QAP is $\min\limits_{\pi\in\Pi(n)}\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}{F[i,j]D[\pi_{i},\pi_{j}]}$, where $F^{n\times n}$ and $D^{n\times n}$ are two input matrices. QAP can be regarded as assigning virtual machines to physical machines, where the transmission matrix $F^{n\times n}$ is defined between virtual machines, and the communication cost matrix $D^{n\times n}$ is defined between physical machines. Comparing with the definition of DRP and DAP, where the transmission matrix is defined between physical machine and virtual machines, it can be noticed that DRP and DAP has a structure more complex than LAP but simpler than QAP. Besides the equivalence of DRP-TOTAL and LAP as well as DRP-BTNK and LBAP, another interesting fact is that the 3-PARTITION problem is used in the reduction to prove the NP-completeness DRP-SSR, while in [56] the 3-PARTITION problem is used to prove that QAP is unapproximable, i.e., there is no polynomial time $r$-approximate algorithm for QAP where $r<\infty$, unless P=NP.

It is worth mentioning that the ROBOT problem defined in [8] has a similar structure with DRP. The input of ROBOT problem includes two functions $f$ and $d$, where $f$ defines the relation between the physical locations and items, and $d$ defines the distance between physical locations. The $f$ and $d$ functions has a similar structure with the $T^{n\times n}$ and $C^{n\times n}$ matrix in DRP problem. The goal of ROBOT problem is to find a TSP with minimal length, where the TSP part makes it NP-complete.

4.3 Number partition problem

The NP-completeness proof for DRP-MSR is based on the PARTITION problem, which is often referred as the number partition problem (NPP) in existing literature. Recall that NPP is given a set $S$ of integers $S=\{s_{1},s_{2},\cdots,s_{n}\}$, and decide whether there exists a partition $(S_{1},S_{2})$ of $S$ s.t. and $\sum\limits_{s_{i}\in S_{1}}s_{i}=\sum\limits_{s_{i}\in S_{2}}s_{i}$. NPP is one of Gary and Johnson’s six basic NP-complete problems [30], and the hardness of this problem is well-known. The approximation for this problem often considers the discrepancy, which is $|\sum\limits_{s_{i}\in S_{1}}s_{i}-\sum\limits_{s_{i}\in S_{2}}s_{i}|$. A famous polynomial time heuristic proposed by Karmarker and Karp [34] is called the differencing method, and can lead to $O(n^{-\alpha\log{n}})$ discrepancy for some positive constant $\alpha$ [61]. Another line of research consider to minimize the discrepancy on randomized data [12, 46, 49]. See [50] for a good survey on NPP. We note there are no existing work trying to design approximate algorithm for the value $\max\left\{\sum\limits_{s_{i}\in S_{1}}s_{i},\sum\limits_{s_{i}\in S_{2}}s_{i}\right\}$ as far as we know.

4.4 Data redistribution

The DRP and DAP problem series emphasizes the importance of data redistribution which have been considered by a lot of former research works. The goal of data redistribution is to minimize the communication cost while satisfying the specific requirement on the data distribution. The works in [17, 53, 57] consider the data redistribution for join operation in database, in which the two papers [53, 57] partially inspire this work. Another work [41] considers the data redistribution in sensor networks. Knoop et al. [36] consider the Distribution Assignment Placement in a engineering point of view, where the abbreviation coincides with DAP in this paper.

5 Future works

Recall that the problem series of DRP and DAP are introduced using parallel sorting and join as the representing example. They reflect the communication pattern of problems that can be solved in one synchronous round, i.e., after one round of communication to redistribute the data, it suffices for all the machines to conduct local computation to finish the computation task. However, there are many problems that need multiple rounds to solve. For example, joining multiple relations can be solved using one round [10] or multiple rounds [1]. Computing the graph coloring [15], maximum matching [26], shortest path [21], etc., must use multiple rounds. The problem will be complicated to minimize the communication cost on WMPC model with multiple rounds. We have the two following observation for future works.

First, it may not be the optimal solution to solve DRP or DAP problem for each round, since the communication of each round is correlated. It also involves to decide a better initial distribution so that the communication cost of the subsequent parallel computation can be reduced. We call this problem the Data Pre-distribution Problem. Second, in this paper it is assumed that each pair of computation machine in WMPC model can communicate in a point-to-point manner, i.e., $C[i,j]<\infty$ for all elements in the communication cost matrix $C$. Actually this assumption is set to be compatible with the one round algorithm, i.e., the machines must be able to reach each other in one round. If multiple rounds are allowed, the assumption of point-to-point communication can be removed, i.e., there can be some element $C[i,j]=\infty$. There will be many interesting but complicated problems such as routing and congestion under the WMPC model, which are left as future work.

6 Conclusion

In this paper we proposed the WMPC (Weighted Massively Parallel Computation) model based on the existing works of topology-aware Massively Parallel Computation model [11, 32]. The WMPC model considers the underlying computation network as a complete weighted graph, which is a complement to the work in [32] where the network topology are restricted to trees. Based on the WMPC model the DRP and DAP problem series are defined, each representing a set of problems with the same pattern of communication. We also defined four kinds of objective functions for communication cost which are TOTAL, BTNK, MSR and SSR, and obtained 8 problems combining the four objective functions with two communication pattern problems. We studied the hardness for the 8 problems, and provided substantial theoretical results. In conclusion, this paper studied the communication minimization problem on WMPC model with a scope both deep and wide, but we must point out that the proposed results only investigated a small portion of the research area on the WMPC or topology-aware MPC model. There are a lot of meaningful problems to be studied following what was studied in this paper.