Redundancy Management for Fast Service (Rates) in Edge Computing Systems

TL;DR

This paper addresses optimizing resource allocation in edge computing to minimize latency and blocking probability through novel algorithms, considering the trade-offs of redundancy in resource-constrained environments.

Contribution

It introduces new algorithms for blocking probability and system time optimization in edge computing, accounting for resource limitations and dynamic system parameters.

Findings

Algorithms outperform benchmarks in simulations

Optimal replication varies with system parameters

Redundancy reduces latency but increases resource contention

Abstract

Edge computing operates between the cloud and end users and strives to provide low-latency computing services for simultaneous users. Redundant use of multiple edge nodes can reduce latency, as edge systems often operate in uncertain environments. However, since edge systems have limited computing and storage resources, directing more resources to some computing jobs will either block the execution of others or pass their execution to the cloud, thus increasing latency. This paper uses the average system computing time and blocking probability to evaluate edge system performance and analyzes the optimal resource allocation accordingly. We also propose blocking probability and average system time optimization algorithms. Simulation results show that both algorithms significantly outperform the benchmark for different service time distributions and show how the optimal replication factor changes with varying parameters of the system.

Tables

Table 1. TABLE I: Optimal Strategy on Average System Time and System Service Rate

	Average System Time							System Service Rate
	Arrival Rate $\lambda$			Cloud Time $𝔼\left[T_{\text{cl}}\right]$			Large $N$	Shift Parameter $\Delta$			Arrival Rate $\lambda$
	Large	Moderate	Small	Large	Moderate	Small		Large	Moderate	Small	Large	Moderate	Small
Exp	C	S	C	S	C&S	C	C	\	\	\	C	C	C
S-Exp	C	S	C	S	C&S	C	C&S	S	C&S	C	S	C&S	C

Equations

$$E[T_{\text{sys}}]=(1-P_b)E[T_{\text{job}}]+P_{b}E[T_{\text{cl}}].$$

$$\sigma =\frac{(1-P_b)}{E[T_{\text{job}}]}+\frac{P_b}{E[T_{\text{cl}}]}.$$

$$E[T_{\text{job}}(m)]=\frac{1}{m\mu }=\frac{c}{N\mu }.$$

$$E[T_{\text{job}}(m)]=\Delta +\frac{1}{m\mu }=\Delta +\frac{c}{N\mu }.$$

$$P_b(c,\rho )=\frac{(\rho {)}^c/c!}{{\sum }_{j=0}^c(\rho {)}^j/j!}$$

$$P_b(c)=\frac{(Kc{)}^c/c!}{{\sum }_{j=0}^c(Kc{)}^j/j!}$$

$$P_b(c)=\frac{(K(\Delta N\mu +c){)}^c/c!}{{\sum }_{j=0}^c(K(\Delta N\mu +c){)}^j/j!}$$

$$P_b(c_1)$$

$$=\frac{1}{{\sum }_{j=0}^{c_1}\frac{c_1!/(c_1-j)!}{(K{c}_1{)}^j}}=\frac{1}{{\sum }_{j=0}^{c_1}{\prod }_{i=1}^j\frac{c_1-j+i}{K{c}_1}}$$

$$<\frac{1}{{\sum }_{j=0}^{c_1}{\prod }_{i=1}^j\frac{c_0-j+i}{K{c}_0}}<\frac{1}{{\sum }_{j=0}^{c_0}\frac{c_0!/(c_0-j)!}{(K{c}_0{)}^j}}$$

$$=\frac{(K{c}_0{)}^{c_0}/c_0!}{{\sum }_{j=0}^{c_0}(K{c}_0{)}^j/j!}=P_b(c_0)$$

$$P_b(c_1)=\frac{(K(\Delta N\mu +c_1){)}^{c_1}/c_1!}{{\sum }_{j=0}^{c_1}(K(\Delta N\mu +c_1){)}^j/j!}$$

$$=\frac{1}{{\sum }_{j=0}^{c_1}\frac{c_1!/j!}{(K(\Delta N\mu +c_1){)}^{c_1-j}}}=\frac{1}{{\sum }_{j=0}^{c_1}({\prod }_{i=1}^j\frac{c_1-j+i}{c_1+\Delta N\mu })\frac{1}{K^j}}$$

$$<\frac{1}{{\sum }_{j=0}^{c_1}({\prod }_{i=1}^j\frac{c_0-j+i}{c_0+\Delta N\mu })\frac{1}{K^j}}<\frac{1}{{\sum }_{j=0}^{c_0}\frac{c_0!/(c_0-j)!}{(K(\Delta N\mu +c_0){)}^j}}$$

$$=\frac{(K(\Delta N\mu +c_0){)}_0^c/c_0!}{{\sum }_{j=0}^{c_0}(K(\Delta N\mu +c_0){)}^j/j!}=P_b(c_0)$$

$$E[T_{\text{sys}}(c)]=(1-P_b(c))E[T_{\text{job}}(c)]+P_b(c)E[T_{\text{cl}}]$$

$$E[T_{\text{sys}}(1)]=(1-P_b(1))\frac{1}{N\mu }+P_b(1)E[T_{\text{cl}}].$$

$$E[T_{\text{sys}}(1)]<E[T_{\text{sys}}(c)]$$

$$\iff \frac{1-P_b(1)}{N\mu }+P_b(1)E[T_{\text{cl}}]\le \frac{1-P_b(c)}{m\mu }+P_b(c)E[T_{\text{cl}}]$$

$$\iff E[T_{\text{cl}}](P_b(1)-P_b(c))\le \frac{1}{\mu }(\frac{1}{m}-\frac{1}{N}+\frac{P_b(1)}{N}-\frac{P_b(c)}{m})$$

$$\iff E[T_{\text{cl}}]\le \frac{1}{\mu }\frac{\frac{1}{m}-\frac{1}{N}+\frac{P_b(1)}{N}-\frac{P_b(c)}{m}}{P_b(1)-P_b(c)}$$

$$\frac{\frac{1}{m}-\frac{1}{N}+\frac{P_b(1)}{N}-\frac{P_b(c)}{m}}{P_b(1)-P_b(c)}\ge 1$$

$$\iff P_b(c)\ge \frac{(1-\frac{1}{N})P_b(1)-\frac{1}{m}+\frac{1}{N}}{1-\frac{1}{m}}$$

$$(1-\frac{1}{N})\frac{K}{1+K}-\frac{1}{m}+\frac{1}{N}\le 0$$

$$\iff K\le \frac{1-\frac{m}{N}}{m-1}=\frac{1-\frac{1}{N}}{m-1}-\frac{1}{N}$$

$$P_b(c)$$

$$=\frac{K^c-K^{c+1}}{1-K^{c+1}}=1-\frac{1-K^c}{1-K^{c+1}}\ge 1-\frac{1}{K}$$

$$1-\frac{1}{K}\ge \frac{(1-\frac{1}{N})\frac{K}{1+K}-\frac{2}{N}+\frac{1}{N}}{1-\frac{2}{N}}$$

$$\iff K\ge N-2\iff \lambda \ge N(N-2)\mu$$

$$E[T_{\text{sys}}(1)]=(1-P_b(1))(\Delta +\frac{1}{N\mu })+P_b(1)E[T_{\text{cl}}].$$

$$E[T_{\text{sys}}(1)]<E[T_{\text{sys}}(c)]$$

$$\iff (1-P_b(1))(\Delta +\frac{1}{N\mu })+P_b(1)E[T_{\text{cl}}]$$

$$\le (1-P_b(c))(\Delta +\frac{1}{m\mu })+P_b(c)E[T_{\text{cl}}]$$

$$\iff E[T_{\text{cl}}](P_b(1)-P_b(c))$$

$$\le \frac{1}{\mu }(\frac{1}{m}-\frac{1}{N}+\frac{P_b(1)}{N}-\frac{P_b(c)}{m})+\Delta (P_b(1)-P_b(c))$$

$$\iff E[T_{\text{cl}}]\le \frac{1}{\mu }\frac{\frac{1}{m}-\frac{1}{N}+\frac{P_b(1)}{N}-\frac{P_b(c)}{m}}{P_b(1)-P_b(c)}+\Delta$$

$$\frac{\frac{1}{m}-\frac{1}{N}+\frac{P_b(1)}{N}-\frac{P_b(c)}{m}}{P_b(1)-P_b(c)}\ge 1$$

$$\iff P_b(c)\ge \frac{(1-\frac{1}{N})P_b(1)-\frac{1}{m}+\frac{1}{N}}{1-\frac{1}{m}}$$

$$(1-\frac{1}{N})\frac{K(\Delta N\mu +1)}{1+K(\Delta N\mu +1)}-\frac{1}{m}+\frac{1}{N}\le 0$$

$$\iff K(\Delta N\mu +1)\le \frac{1-\frac{m}{N}}{m-1}=\frac{1-\frac{1}{N}}{m-1}-\frac{1}{N}$$

Full text

I Introduction

With the rapid increase in IoT applications, such as smart cities and homes, autonomous vehicles, and artificial intelligence, billions of IoT devices are coming into our everyday lives [2, 3]. The demand for low latency storage and computing services is increasing to accommodate novel IoT platforms (e.g., deep learning) [4, 5, 6]. Some applications, for example, connected and autonomous vehicles, smart healthcare, and ocean monitoring require fast or no service. Cloud services are inefficient at responding to such applications.

Edge computing is an inter-layer between the cloud and the end-user. It provides storage and computing infrastructure at the node located one or two network hops from the end-user [7]. According to [8], the round trip time between the cloud and end-user (about $17.989$ ms) is over $10$ times larger than the time between the edge and end-user (about $1.416$ ms). Therefore, in the edge system, the bottleneck of the computing service is no longer the communication delay. As illustrated in Fig. 1, an edge computing system receives and processes jobs from end-users, and on a much smaller scale of storage and computing resources than the cloud [9, 10]. Service requests get sent to the cloud when all edge workers are busy, and communication time becomes a significant part of the delay.

Edge nodes often operate in highly unpredictable environments and handle jobs needing fast or no service. Using resources redundantly is a known strategy to enable fast service. However, with increased resource usage comes a decrease in the number of jobs the edge system can execute simultaneously. Thus, more jobs get sent to the cloud, which increases their execution time. An edge computing system that sends jobs to the cloud once the resources become unavailable acts locally as a blocking system. The jobs sent to the cloud will experience higher latency as they forgo the geographic benefit [11].

We, therefore, need to address two opposing objectives to maximize the benefits of edge computing. The first objective is to minimize the computing speed of each job executed locally at the edge. (Computing speed, expressed in terms of computing time or service rate, is a classical and essential performance metric [12, 13].) The second objective is to minimize the number of jobs processed by the edge system (i.e., the number of blocked jobs sent to the cloud. Due to the limited storage and computing resources, the edge system may be unable to process all jobs independently of the cloud.

We focus on edge computing systems in which a single controller node manages a computing cluster of workers [14]. When a job arrives, the controller replicates it to several workers. Here, we refer to the workers processing the same job as the replication group and its size as the replication factor. Since the total number of workers in the system is limited, we have to decrease the number of groups when we increase the replication factor. The edge computing system with a higher replication factor may have a higher blocking probability. That is, it can process fewer jobs simultaneously. Therefore, processing more jobs locally may decrease the expected computing speed.

Simultaneously minimizing the local job computing time and job blocking probability is generally impossible. Since both are crucial to edge computing, we will focus on finding the tradeoff between these performance metrics. We combine the local edge computing time and the time spent to reach the cloud (which happens with the job-blocking probability at the edge) into a single metric representing the average service time. Recent literature has considered the service rate a critical performance metric, especially in distributed storage systems. For example, [15] evaluates the service rate of a distributed storage system to find an optimal storage allocation strategy. [16] uses the service rate region as an essential consideration in the design of erasure-coded distributed systems. In some scenarios, we will show the benefits of using the service rate instead of the expected time. We will use both metrics to evaluate the tradeoff between the local job computing time and the blocking probability at the edge.

In this paper and [1], we initiate a study of the value of using replication in edge-blocking systems to improve their service rate. (We remind the reader that there are other related crucial edge performance metrics, such as offload latency, power consumption, and energy efficiency, [17, 18, 19, 20], which are beyond the scope of this paper. ) A vast body of literature argues the benefits of redundancy in distributed computing (see, e.g., [21, 22], and references therein), including edge systems [23, 24, 25, 26]. However, because of the adverse effects of temporarily redundantly using more system resources, there are limits to these benefits. The optimal levels of redundancy have not been fully characterized; see, e.g., [27, 28] and references therein, and are the subject of current research.

The paper is organized as follows. In Sec. II, we describe the architecture of the edge computing system and formulate the problem. In Sec. III, we state the problem and summarize the contributions of this paper. In Sec. IV, we theoretically and numerically analyze the job computing time and the job blocking probability changes with the number of groups. In Sec. V and VI, we analyze the optimal number of groups (or replication factor) that minimizes the average system time considering different values of $\lambda$ and $N$. In Sec. VII, we maximize the system service rate for the scenario that the time a job spends in the cloud is long. The conclusions are given in Sec. VIII.

II System Model

II-A System Architecture

We consider the edge computing system model shown in Fig. 2 as a combination of the distributed computing system and the blocking system. The edge computing system has limited storage and computing resources[3]. It consists of a single front-end controller node and multiple computing servers, which we refer to as workers. The single controller node manages the entire computing cluster of nodes. This architecture is commonly implemented in modern frameworks, such as Apache Mesos [29], and edge computing systems with limited storage and computing resources [14].

The controller node will divide the workers into several groups. The controller node creates $m$ copies for each arriving job and assigns each copy to a worker in a group. In Fig. 2, the controller node sends $J_{1}$ and its copy to workers 1 and 2 ($m=2$), and sends $J_{2}$ and its copy to workers 3 and 4 ($m=2$). This execution replication mitigates straggling and reduces the job’s expected completion time. The larger the replication factor $m$, the higher the reduction of the job’s average completion time.

On the other hand, when all workers are busy, the new request for job execution gets blocked. An edge computing system may send such jobs to the cloud (see Fig. 2, job $J_{5}$). There is a significant communication delay between the edge and the cloud, which may be much longer than the expected job computing time. Therefore, we want the system to serve more jobs and send fewer to the cloud.

II-B Redundancy and Straggler Mitigation

In an edge computing system, the job computing time $T_{\text{job}}$ is a crucial performance metric. However, task straggling is a fundamental problem in distributed systems that significantly affects system performance. To solve this problem, replication is an effective technique to introduce redundancy to mitigate stragglers. In Fig. 2, the controller node sends the $J_{1}$ and its copy to workers 1 and 2 separately. Compared to $J_{2}$ without redundancy, even though one worker processes the job for a long time, the controller node can still receive the result from the other worker. This problem is well studied in [28].

II-C Traditional Performance Metrics

There are two performance metrics of interest in the described system: 1) the job computing time $T_{\text{job}}$ and 2) the job blocking probability $P_{b}$. The system’s goal is to minimize both of these numbers. The design parameters for a given system size (fixed number of workers) are the replication factor $m$ and the number of groups $c$. Increasing $m$ and decreasing $c$ reduce $T_{\text{job}}$ (improves the first performance metric). However, the effect of $m$ and $c$ on $P_{b}$ is not immediately apparent. Increasing $m$ will temporarily occupy more servers per job, but will also make the jobs stay in the system for a shorter time.

II-D * Average Time and Service Rate as Performance Metrics*

In general, there may not be an optimal replication factor $m$ that simultaneously minimizes both $\operatorname{\mathbb{E}}[T_{\text{job}}]$ and $P_{b}$. We use the * average system time* to evaluate the tradeoff between these above two metrics. The average system time is defined as

[TABLE]

where $T_{\text{cl}}$ is the completion time of the jobs blocked by the edge system and executed in the cloud.

Except for the average system time, we combine the above two metrics to get a new metric we refer to as the system service rate and define it as follows:

[TABLE]

We consider the average system time as the main performance metric to evaluate the tradeoff. However, according to the results in Sec. V, the average system time does not perform well when $\operatorname{\mathbb{E}}[T_{\text{cl}}]\gg\operatorname{\mathbb{E}}[T_{\text{job}}]$. Then, the system service rate is the better metric to evaluate the tradeoff.

II-E Job Arrival and Service Time

M/M/1 or M/M/n queues have been used in task allocation models in edge systems[30, 31, 32]. Here, we assume the job arrivals follow a Poisson process with a rate $\lambda$, which allows us to model the blocking system as an M/G/c/k queue. Analyzing the blocking probability of an M/G/c/k queue that models the blocking operation of the system in Fig. 2 is a highly complex problem. To better understand it, we first consider the Erlang B model, the M/M/c/c queue, where the queue length equals the number of groups. Then the service time follows the exponential distribution $\operatorname{Exp}(\mu)$, which is a straightforward model and widely used in distributed computing[33, 34, 35]. Similarly, we will also consider the M/G/c/c queue with the shifted exponential $\operatorname{S-Exp}(\Delta,\mu)$ service time. Shifted exponential distribution is also widely used in distributed computing[36, 34, 37]. Here, $\Delta$ is an initial handshake time, after which the worker will complete the job in some $\operatorname{Exp}(\mu)$ time.

III Problem Statement and Contributions

[TABLE]

System parameters and notations are summarised in the above list. Our goal is to evaluate two performance metrics, the expected job computing time $\operatorname{\mathbb{E}}[T_{\text{job}}]$ and the job blocking probability $P_{b}$ for systems with Poisson job arrivals and (shifted) exponential service time. We separately compute the number of groups $c$ (or the replication factor $m$) that minimizes $\operatorname{\mathbb{E}}[T_{\text{job}}]$ and $P_{b}$. When $c$ is small, the system concentrates limited computing resources on a few jobs. When $c$ is large, the system spreads the computing resources to more jobs. Apparently, the job will be processed faster with more computing resources and slower with less resources. To better evaluate $\operatorname{\mathbb{E}}[T_{\text{job}}]$ and $P_{b}$, we use the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ and the system service rate $\sigma$ to evaluate the tradeoff between these two metrics. We show how we can achieve the desired tradeoff between these two traditional metrics by selecting an appropriate $c$.

We summarize the optimal strategy on the average system time in Table I. We conclude that it is better to spread computing resources to more jobs when the job arrival rate is moderate and the cloud time is large. Otherwise, we should consider to concentrate computing resources on a few jobs. When the computing resources are unlimited, it is always better to concentrate them on a few jobs for the exponential service time. However, we should find the balance between concentrating and spreading the computing resources for the shifted exponential service time.

We also summarize the optimal strategy on the system service rate in Table I for the scenario with large cloud time. We conclude that it is better to concentrate computing resources on a few jobs with exponential service time. We should spread computing resources to more jobs when the shifted parameter and job arrival rate are large. Otherwise, we should concentrate computing resources on a few jobs.

We here put the relevant results of [1], which focuses on analyzing the blocking system with blocking probability and system service rate. Further contributions include the following:

• •

We propose an edge computing system model as a combination of the distributed computing system and the blocking system. We aim to find the optimal number of groups (or replication factor) that minimizes the job computing time and the job blocking probability under different service time distributions. The theoretical and numerical results show that job computing time increases with decreasing replication, and the system blocking probability increases with the replication factor.

• •

We adopt the average system time to evaluate the tradeoff between job computing time and the system blocking probability. We analyze the optimal number of groups that maximizes the system service rate for different scenarios. We conclude that the optimal number of groups changes with the computing time distribution parameters and the arrival rate.

• •

We propose a new performance metric, the system service rate, to evaluate the tradeoff for the scenario when the time spent in the cloud is long. Our theoretical and numerical results show that concentrating limited computing resources on each job is better scheduling.

IV Computing Time and Blocking Probability

IV-A Job Computing Time

The job computing time measures how much time the job spends in the system occupying resources. We consider the system uses $m$-fold replication, and the worker computing time follows (shifted) exponential distribution. When the worker computing time follows $\operatorname{Exp}(\mu)$, the expected job computing time is given by

[TABLE]

Observe that $\operatorname{\mathbb{E}}[T_{\text{job}}]$ reaches its minimum at $m=1$. Moreover, $T_{\text{job}}$ follows the exponential distribution with the rate parameter $m\mu$. When the worker computing time follows $\operatorname{S-Exp}(\Delta,\mu)$, the expected job computing time is given by

[TABLE]

Observe that $\operatorname{\mathbb{E}}[T_{\text{job}}]$ reaches its minimum at $m=1$. Moreover, $T_{\text{job}}$ follows the shifted exponential distribution with the shift $\Delta$ and the rate parameter $m\mu$. See, e.g., [38]. The above conclusions show that increasing the computing resource for each job will surely decrease the job computing time. Next, we want to explore if this strategy will perform well considering the blocking probability.

IV-B Job Blocking Probability

We adopt the (shifted) exponential distribution as a classical and simple service time model to analyze the edge computing system. Analyzing the blocking probability of an M/M/c/k (or M/G/c/k) queue that models the blocking operation of the system in Fig. 2 is a highly complex problem. To better understand it, we consider the Erlang B model, the M/M/c/c (or M/G/c/c) queue, where the queue length equals the number of groups. Then, for a blocking system with $c$ groups and the job arrives as a Poisson process with the rate $\lambda$, the job blocking probability is

[TABLE]

where $\rho=\lambda\operatorname{\mathbb{E}}[T_{\text{job}}]$. The above expression shows that for a given $c$, $P_{b}$ increases with $\rho$. From (3), we know that $\operatorname{\mathbb{E}}[T_{\text{job}}]$ is a function of $m$, in which $m=\frac{N}{c}$. Then we will take $K=\frac{\lambda}{N\mu}$. Thus, $P_{b}$ is a function of $c$. When the worker computing time follows $\operatorname{Exp}(\mu)$, we can rewrite (5) in the following,

[TABLE]

where $K=\frac{\lambda}{N\mu}$ is a constant. When the worker computing time follows $\operatorname{S-Exp}(\Delta,\mu)$, we can rewrite (5) in the following,

[TABLE]

According to (6), We find the optimal $c$ minimizes the job blocking probability $P_{b}$ in Theorem 1.

Theorem 1**.**

For the blocking system with Poisson($\lambda$) arrivals and $\operatorname{Exp}(m\mu)$ service time, the job blocking probability $P_{b}$ increases with the number of groups $c$ and reaches the minimum at $c=N$ (i.e., $m=1$).

Proof.

Assume $c_{1}>c_{0}$, from (6), the blocking probability is

[TABLE]

Since $c\in[1,N]$ , $P_{b}$ reaches its minimum at $c=N$. ∎

According to (7) and Theorem 1, we have the following lemma that finds the optimal $c$ minimizes the job blocking probability $P_{b}$ for shifted exponential service time.

Lemma 1**.**

For the blocking system with Poisson($\lambda$) arrivals and $\operatorname{S-Exp}(\Delta,m\mu)$ service time, the job blocking probability $P_{b}$ increases with the number of groups $c$ and reaches the minimum at $c=N$ (i.e., $m=1$).

Proof.

Similar to the proof of Theorem 1, assume $c_{1}>c_{0}$, from (7), the blocking probability is

[TABLE]

Since $c\in[1,N]$, $P_{b}$ increases with the number of groups $c$ and reaches its minimum at $c=N$. ∎

From Theorem 1 and Lemma 1, we conclude that spreading the computing resources to more jobs is the better way to support the system processing more jobs. Therefore, optimizing the blocking probability and the expected job computing time with the variable $c$ is a dilemma.

IV-C Numerical Analysis

We evaluate (3) and (6) for $\operatorname{\mathbb{E}}[T_{\text{job}}]$ vs. $c$ and $P_{b}$ vs. $c$ in Fig. 3. Meanwhile, we evaluate (4) and (7) for $\operatorname{\mathbb{E}}[T_{\text{job}}]$ vs. $c$ and $P_{b}$ vs. $c$ in Fig. 4. We consider a system with $N=24$ workers, the job arrives following the Poisson distribution with the rate $\lambda=1$.

In Fig. 3, the service time for each worker follows the exponential with $\mu=0.1$. The left subfigure shows that $\operatorname{\mathbb{E}}[T_{\text{job}}]$ increases linearly with $c$, which means that introducing more redundancy provides a higher computing speed. The right subfigure shows that $P_{b}$ decreases with increasing $c$, which means introducing more redundancy leads to more jobs sent to the cloud. We also observe that $P_{b}$ is close to [math] and decreases slowly when $c$ is large; when $c\leq 6$, $P_{b}$ decreases sharply. These observations are consistent with the theoretical analysis in Theorem 1. Meanwhile, We also conclude that some minimal replication significantly reduces computing time with almost no blocking probability change.

In Fig. 4, the service time for each worker follows the shifted exponential with $\Delta=5$ and $\mu=0.1$. We observe that $\operatorname{\mathbb{E}}[T_{\text{job}}]$ increases linearly with $c$ and $P_{b}$ decreases with increasing $c$. We also observe that $P_{b}$ is close to [math] only when $c$ is large enough, and it does not decrease sharply compared to the results in Fig.3. These observations are consistent with the theoretical analysis in Lemma 1.

The above two figures clearly show that minimizing the job computing time and the job blocking probability is impossible. Considering both job computing time and job blocking probability, we find that each metric requires a very different optimal $c$. However, the job blocking probability may not always decrease with the number of groups for some heavy tail distributions, e.g., Pareto distribution. We will explore this problem in future work.

V Average System Time

In Section IV, we know that simultaneously minimizing the job computing time and the blocking probability is impossible. Interestingly, some minimal replication significantly reduces computing time with almost no blocking probability change. Therefore, we adopt the average system time, as a single performance indicator describing the tradeoff between the job computing time and the job blocking probability. For a system with $N$ workers and $c$ groups, the expression of the average system time is

[TABLE]

In this paper, we assume that the edge system can provide a better computing performance by reducing the communication time between the users and the cloud. When $\operatorname{\mathbb{E}}[T_{\text{job}}(c)]\geq\operatorname{\mathbb{E}}[T_{\text{cl}}]$, it is easy to prove that $\operatorname{\mathbb{E}}[T_{\text{sys}}(c)]\geq\operatorname{\mathbb{E}}[T_{\text{cl}}]$. This means that the edge system does not have enough computing resources for each job. Then we have the following claim.

Claim 1**.**

If $\operatorname{\mathbb{E}}[T_{\text{cl}}]\leq\operatorname{\mathbb{E}}[T_{\text{job}}]$, the job should always be sent to the cloud.

From (3) (or (4)), the edge system will spend $\frac{1}{\mu}$ (or $\Delta+\frac{1}{\mu}$) time to process the job with minimum computing resource, and $\frac{1}{N\mu}$ (or $\Delta+\frac{1}{N\mu}$) time to process the job with a maximum computing resource. Therefore, Claim 1 holds for (shifted) exponential distributions. In the following, we will consider the scenarios when $\operatorname{\mathbb{E}}[T_{\text{cl}}]>\operatorname{\mathbb{E}}[T_{\text{job}}(c=N)]$.

V-A Cloud Time: $\operatorname{\mathbb{E}}[T_{\text{job}}(c=N)]>\operatorname{\mathbb{E}}[T_{\text{cl}}]>\operatorname{\mathbb{E}}[T_{\text{job}}(c=1)]$

When cloud time satisfies $\operatorname{\mathbb{E}}[T_{\text{cl}}]>\operatorname{\mathbb{E}}[T_{\text{job}}(c=1)]$, it is faster to process the job in the edge system. Then, we should use the edge system to reduce the average system time. That is, the main purpose of this section is to find the optimal $c$ (or $m$) that minimizes the $\operatorname{\mathbb{E}}[T_{\text{sys}}]$.

First, we consider the exponential service time. The cloud time condition is $\operatorname{\mathbb{E}}[T_{\text{cl}}]\in(\frac{1}{N\mu},\frac{1}{\mu})$. According to Claim 1, when $m<\frac{1}{\mu\operatorname{\mathbb{E}}[T_{\text{cl}}]}$, the job should be sent to the cloud; otherwise, the job should be sent to the edge. That is, the optimal $m$ should be in the region $(\frac{1}{\mu\operatorname{\mathbb{E}}[T_{\text{cl}}]},N)$. In the following theorem, we find the conditions that the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=1$ ($m=N$).

Theorem 2**.**

For the edge system with Poisson($\lambda$) arrivals and $\operatorname{Exp}(\mu)$ computing time, when $\operatorname{\mathbb{E}}[T_{\text{cl}}]\in(\frac{1}{N\mu},\frac{1}{\mu})$, the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=1$ ($m=N$) under the conditions $\lambda\geq N(N-2)\mu$ or $\lambda\leq\frac{N\mu}{N-2}$.

Proof.

When $c=1$ and $m=N$, the average system time is

[TABLE]

Then, for any $c\geq 2$ we have

[TABLE]

Let the above inequality holds, we can have

[TABLE]

Since $P_{b}(c)\geq 0$, the inequality (9) holds when $(1-\frac{1}{N})P_{b}(1)-\frac{1}{m}+\frac{1}{N}\leq 0$. As we know $P_{b}(1)=\frac{K}{1+K}$, then we have

[TABLE]

Apparently, the right-hand side increases with decreasing $m$. Therefore, $K\leq\frac{1}{N-2}$, that is, $\lambda\leq\frac{N\mu}{N-2}$.

According to (5),

[TABLE]

Then the inequality (9) holds when $1-\frac{1}{K}\geq((1-\frac{1}{N})P_{b}(1)-\frac{1}{m}+\frac{1}{N})/(1-\frac{1}{m})$. The right-hand side reaches the maximum at $m=N/2$, therefore we have

[TABLE]

Therefore, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=1$ ($m=N$) under the conditions that $\lambda\geq N(N-2)\mu$ or $\lambda\leq\frac{N\mu}{N-2}$. ∎

Next, we consider the shifted exponential service time. The cloud time condition is $\operatorname{\mathbb{E}}[T_{\text{cl}}]\in(\Delta+\frac{1}{N\mu},\Delta+\frac{1}{\mu})$. Similarly, when $m\leq\frac{1}{\mu\operatorname{\mathbb{E}}[T_{\text{cl}}]}$, the job should be sent to the cloud; otherwise, the job should be sent to the edge. That is, the optimal $m$ should be in the region $(\frac{1}{\mu\operatorname{\mathbb{E}}[T_{\text{cl}}]},N)$. According to Theorem 2, we can find the conditions that the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=1$ ($m=N$) in the following lemma.

Lemma 2**.**

For the edge system with Poisson($\lambda$) arrivals and $\operatorname{S-Exp}(\Delta,\mu)$ computing time, when $\operatorname{\mathbb{E}}[T_{\text{cl}}]\in(\Delta+\frac{1}{N\mu},\Delta+\frac{1}{\mu})$, the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=1$ ($m=N$) under the conditions $\lambda\geq\frac{N(N-2)\mu}{\Delta N\mu+1}$ or $\lambda\leq\frac{N\mu}{(N-2)(\Delta N\mu+1)}$.

Proof.

When $c=1$ and $m=N$, the average system time is

[TABLE]

Then, for any $c\geq 2$ we have

[TABLE]

Since $\operatorname{\mathbb{E}}[T_{\text{cl}}]\leq\Delta+\frac{1}{\mu}$, then the above inequality holds when

[TABLE]

Similarly to the proof of Theorem 2, the above inequality holds when $(1-\frac{1}{N})P_{b}(1)-\frac{1}{m}+\frac{1}{N}\leq 0$. Then we have

[TABLE]

Since $m\leq\frac{N}{2}$. Therefore, $K\leq\frac{1}{(N-2)(\Delta N\mu+1)}$, that is, $\lambda\leq\frac{N\mu}{(N-2)(\Delta N\mu+1)}$.

Then we consider the second case.

[TABLE]

We have $\operatorname{\mathbb{E}}[T_{\text{sys}}(1)]\leq\operatorname{\mathbb{E}}[T_{\text{sys}}]$ when $1-\frac{1}{K(\Delta N\mu+1)}\geq\frac{(1-\frac{1}{N})P_{b}(1)-\frac{1}{m}+\frac{1}{N}}{1-\frac{1}{m}}$. The right-hand side reaches the maximum at $m=N/2$, therefore we have $K(\Delta N\mu+1)\geq N-2$, that is, $\lambda\geq\frac{N(N-2)\mu}{(\Delta N\mu+1)}$ ∎

From Theorem 2 and Lemma 2, it is obvious that the optimal $c$ is determined by the ratio of $\lambda$ and $\mu$. When $\lambda$ is small or large, it is better to concentrate the limited computing resources on a few jobs. Otherwise, the system can properly spread computing resources to more jobs. However, since the restriction of Claim 1, the optimal $c\leq\frac{N}{2}$.

V-B * Cloud Time: $\operatorname{\mathbb{E}}[T_{\text{cl}}]\geq\operatorname{\mathbb{E}}[T_{\text{job}}(c=N)]$*

First, we consider the exponential service time. The cloud time condition is $\operatorname{\mathbb{E}}[T_{\text{cl}}]\geq\frac{1}{\mu}$. In this scenario, the edge system is always the first choice for each job. The job will only be sent to the cloud when the edge system is busy. Similarly to the conclusion of Theorem 2, the optimal $c$ that minimizes the average system time changes with different system parameters. However, we can still draw certain conclusions from the following theorem.

Theorem 3**.**

For the edge system with Poisson($\lambda$) arrivals and $\operatorname{Exp}(\mu)$ job computing time, when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is sufficiently large, the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=N$ ($m=1$).

Proof.

When $c=N$ and $m=1$, the average system time is

[TABLE]

Then, for any $c<N$ we have

[TABLE]

Apparently, the right-hand side of the above inequality is finite. Therefore, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=N$ ($m=1$) when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is sufficiently large. ∎

Next, we consider the shifted exponential service time. The cloud time condition is $\operatorname{\mathbb{E}}[T_{\text{cl}}]\geq\Delta+\frac{1}{\mu}$. According to the proof of Theorem 3, we can also prove in the following lemma that the system should spread the computing resources to more jobs when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is sufficiently large.

Lemma 3**.**

For the edge system with Poisson($\lambda$) arrivals and $\operatorname{S-Exp}(\Delta,\mu)$ job computing time, when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is sufficiently large, the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=N$ ($m=1$).

Proof.

When $c=N$ and $m=1$, the average system time is

[TABLE]

Then, for any $c<N$ we have

[TABLE]

Apparently, the right-hand side of the above inequality is finite. Therefore, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches the minimum at $c=N$ ($m=1$) when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is sufficiently large. ∎

It is not surprising to see the conclusion of Theorem 3 and Lemma 3. Because when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is sufficiently large, minimizing $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ is almost equal to minimizing $P_{b}(c)\operatorname{\mathbb{E}}[T_{\text{cl}}]$. However, under this condition, the average system time may not be a good parameter to evaluate the tradeoff between the job computing time and the blocking probability. Therefore, we may consider another system parameter for this scenario in Section VII.

V-C Numerical Analysis

We evaluate (8) to see how the average system time changes with $c$ in a system with $N=24$ workers. We consider two service time models: the exponential service time with $\mu=0.1$ and the shifted exponential service time with $\mu=0.1$ and $\Delta=5$.

, the job arrives following the Poisson distribution with different values of $\lambda\in\{0.1,1,10,40\}$, and the expected cloud time $\operatorname{\mathbb{E}}[T_{\text{cl}}]=8$ is smaller than the maximum job computing time.

In Fig. 5, we consider $\operatorname{\mathbb{E}}[T_{\text{cl}}]=8$ is smaller than the maximum job computing time and plot $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ vs. $c$ for two service time models with $\lambda\in\{0.1,1,10,40\}$. The upper graph shows when $\lambda=0.1$ is small enough, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches its minimum at $c=1$, which is consistent with the result in Theorem 2. To decrease $c$ and increase the replication factor $m$ will significantly reduce $\operatorname{\mathbb{E}}[T_{\text{sys}}]$. When $\lambda=1$ or $10$, we should slightly increases $c$ to achieve a lower $\operatorname{\mathbb{E}}[T_{\text{sys}}]$. When $\lambda=40$ is sufficiently large, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches its minimum at $c=1$ again. However, the optimal value of $c$ does not provide a significant reduction of $\operatorname{\mathbb{E}}[T_{\text{sys}}]$. We can observe similar results in the lower graph. Compared with the upper graph, we observe that $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ increases significantly when $\lambda$ is small (e.g., $\lambda=0.1$ or $1$) and almost remains the same when $\lambda$ is large (e.g., $\lambda=10$ or $40$). From the results, we conclude that when fewer jobs arrive, almost all jobs can be processed in the edge system, so it is better to increase the computing speed. When more jobs arrive, the system should reduce the computing speed and provide the computing resources for more jobs. When there are too many jobs, the resource allocation strategy can not change the system performance significantly.

In Fig. 6, we consider the job arrives following the Poisson distribution with $\lambda=2$, and plot $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ vs. $c$ for two service time models with $\operatorname{\mathbb{E}}[T_{\text{cl}}]\in\{15,50,100\}$. The upper graph shows when $\operatorname{\mathbb{E}}[T_{\text{cl}}]=15$, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches its minimum at $c=4$; When $\operatorname{\mathbb{E}}[T_{\text{cl}}]=50$, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches its minimum at $c=12$; When $\operatorname{\mathbb{E}}[T_{\text{cl}}]=100$, $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ reaches its minimum at $c=24$. We also observe that the optimal $c$ increases with $\operatorname{\mathbb{E}}[T_{\text{cl}}]$. We can observe similar results in the lower graph. Compared with the upper graph, we observe that $c=24$ can be optimal with a smaller value of $\operatorname{\mathbb{E}}[T_{\text{cl}}]$. From the results, we have the following conclusions. Generally, we should balance the tradeoff between the job computing time and the blocking probability. However, when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is sufficiently large, we should only focus on reducing the blocking probability and assigning resources for more jobs.

VI Average System Time with Large $N$

Since the complexity of the expression of the average system time, it is difficult to find optimal $m$ and $c$. The above section concludes that the optimal $m$ and $c$ change with the arrival rate $\lambda$ and the parameters of $\operatorname{Exp}$ or $\operatorname{S-Exp}$ service time. However, it is not enough to analyze the overall performance of the system. In practice, some edge systems may have more computing resources. Therefore, it is valuable to analyze how the optimal $m$ and $c$ change when $N$ is large. In this section, we assume $N\to\infty$. Since $N=mc$, there are three possible scenarios:

1. Scenario 1:

$m\to\infty$ and $c$ is finite. 2. Scenario 2:

$c\to\infty$ and $m$ is finite. 3. Scenario 3:

both $m$ and $c$ go to infinity.

Apparently, the above three scenarios can not exist simultaneously and only one of them provides the best performance considering the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$. We will evaluate the performance of each scenario under different service time distributions in the following.

VI-A Analysis of Scenario 1

In Scenario 1, since $c$ is finite, there exists a constant $c_{0}$ that $c$ is always smaller than $c_{0}$. First, we consider the exponential service time. We find the minimum average system time in the following theorem.

Theorem 4**.**

In the edge system with $\operatorname{Exp}(\mu)$ service time, as the number of workers $N$ goes to infinity, when the number of groups $c$ is finite, the average time reaches the minimum at [math].

Proof.

We adopt the expression of the average system time from (8) and replace $m$ with $N/c$, then

[TABLE]

Since we assume that $c$ is finite, then we have

[TABLE]

Since $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is a constant,

[TABLE]

Therefore, when $N$ goes to infinity and $c$ is finite, the average system time will reduce to [math]. ∎

From Theorem 4, we know that the average system time $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ can reach the minimum at [math]. Then we can conclude that the system with exponential service time can get the best performance by adopting Scenario 1.

Next, we consider the shifted exponential service time. According to the proof of Theorem 4, we find the minimum average system time for the shifted exponential service time in the following lemma.

Lemma 4**.**

In the edge system with $\operatorname{S-Exp}(\Delta,\mu)$ service time, as the number of workers $N$ goes to infinity, when the number of groups $c$ is finite, the average time reaches the minimum at $\Delta+\frac{(\operatorname{\mathbb{E}}[T_{\text{cl}}]-\Delta)(\lambda\Delta)^{c_{0}}/c_{0}!}{\sum_{j=0}^{c_{0}}(\lambda\Delta)^{j}/j!}$.

Proof.

We adopt the expression of the average system time from (8) and replace $m$ with $N/c$, then

[TABLE]

Since we assume that $c$ is finite, then we have

[TABLE]

Since $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is a constant and $c\leq c_{0}$,

[TABLE]

∎

Compared to Theorem 4, we can not decide if the system with shifted exponential service time can reach the best performance by adopting Scenario 1.

VI-B Analysis of Scenario 2

In Scenario 2, when $m$ is finite, that is, there exists a constant $m_{0}$ that $m$ is always smaller than $m_{0}$. First, we consider the exponential service time. We find the minimum average system time in the following theorem.

Theorem 5**.**

In the edge system with $\operatorname{Exp}(\mu)$ service time, as the number of workers $N$ goes to infinity, when the replication factor $m$ is finite, the average time reaches the minimum at $\frac{1}{m_{0}\mu}$.

Proof.

We adopt the expression of the average system time from (8), then

[TABLE]

Since we assume that the $m$ is finite, then we have

[TABLE]

Since $m$ is finite, $\frac{\lambda}{m\mu}$ is also finite. Then, we have

[TABLE]

Since $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is a constant,

[TABLE]

Therefore, when $N$ goes to infinity and $m$ reaches the maximum at $m_{0}$, the average system time reaches the minimum at $\frac{1}{m_{0}\mu}$. ∎

Comparing the results of Theorem 4 and 5, we can conclude that the system will not adopt Scenario 2 considering the average system time.

Next, we consider the shifted exponential service time. According to the proof of Theorem 5, we find the minimum average system time for shifted-exponential service time in the following lemma.

Lemma 5**.**

In the edge system with $\operatorname{S-Exp}(\Delta,\mu)$ service time, as the number of workers $N$ goes to infinity, when the replication factor $m$ is finite, the average time reaches the minimum at $\Delta+\frac{1}{m_{0}\mu}$.

Proof.

We adopt the expression of the average system time from (8), then

[TABLE]

Since $m$ and $\Delta$ are finite, $\lambda\Delta+\frac{\lambda}{m\mu}$ is also finite. Then, we have

[TABLE]

and

[TABLE]

Thus,

[TABLE]

Since $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is a constant and $m\leq m_{0}$,

[TABLE]

∎

Compared to the results of Lemma 4, we can not easily decide which is smaller. Thus, we will make an overall comparison after calculating the average system time with Scenario 3.

VI-C Analysis of Scenario 3

In Scenario 3, the replication factor $m$ and the number of groups $c$ go to infinity. First, we consider the exponential service time. We find the minimum average system time with Scenario 3 in the following theorem.

Theorem 6**.**

In the edge system with $\operatorname{Exp}(\mu)$ service time, when the replication factor $m$ and the number of groups $c$ go to infinity, the average time reaches the minimum at [math].

Proof.

We adopt the expression of the average system time from (8), then

[TABLE]

First, since all the parameters are positive, we have

[TABLE]

Then we have

[TABLE]

It is obvious that

[TABLE]

Thus, we have

[TABLE]

That is,

[TABLE]

Therefore, when $c$ and $m$ go to infinity, the average system time reaches the minimum at [math]. ∎

From Theorem 6, we find that $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ can also reach the minimum at [math]. Then we conclude that the system can get the best performance by adopting Scenario 3. Compared to the results of Theorem 4, we find that it does not matter whether the number of groups $c$ is finite or infinite. To achieve the best system performance, we only need to let the replication factor $m$ increase with $N$.

Next, we consider the shifted exponential service time. According to the proof of Theorem 6, we find the minimum average system time for the shifted exponential service time in the following lemma.

Lemma 6**.**

In the edge system with $\operatorname{S-Exp}(\Delta,\mu)$ service time, when the replication factor $m$ and the number of groups $c$ go to infinity, the average time reaches the minimum at $\Delta$.

Proof.

We adopt the expression of the average system time from (8), then

[TABLE]

First, since all the parameters are positive, we have

[TABLE]

Then we have

[TABLE]

As $c$ goes to infinity, $c!$ increases much faster than $(\lambda\Delta+\frac{\lambda}{\mu})^{c}$. Thus, we have

[TABLE]

That is,

[TABLE]

∎

Finally, we can compare the minimum values of $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ from Lemma 4,5 and 6, it is obvious that the minimum average system time with Scenario 3 reaches the overall minimum at $\Delta$. Therefore, to reach the best system performance, we should let both the replication factor $m$ and the number of groups $c$ increase with $N$.

VI-D * Numerical Analysis*

We evaluate (3) and (8) to find the optimal $c$ and $m$ minimize the average system time. Then we separately plot the optimal $c$ vs. $N$ and the optimal $m$ vs. $N$ for exponential service time in Fig. 7. Similarly, we separately plot the optimal $c$ vs. $N$ and the optimal $m$ vs. $N$ for shifted exponential service time in Fig. 8 by evaluating (4) and (8). We consider a system with $N=24$ workers, the job arrives following the Poisson distribution with the rate $\lambda=5$, the expected time a job spends in the cloud is $\operatorname{\mathbb{E}}[T_{\text{cl}}]=10$. Since $m$ and $c$ are integers, we assume $m=\lfloor\frac{N}{c}\rfloor$. Although the actual values of $\operatorname{\mathbb{E}}[T_{\text{sys}}]$ may become small, it does not affect the conclusions of the numerical analysis.

In Fig. 7, the service time for each worker follows the exponential with $\mu=0.1$. We observe that when $N$ is small (e.g., $N\leq 40$), the optimal $c$ increases with $N$; when $N$ becomes large (e.g., $N>40$), the optimal $c$ decreases with increasing $N$. We also observe that the optimal $m$ always increases with $N$ except for the scenario when $N$ is small. These observations are consistent with the theoretical analysis of Theorem 4,5 and 6.

In Fig. 8, the service time for each worker follows the shifted exponential with $\Delta=5$ and $\mu=0.1$. We observe that the optimal $c$ increases sharply with $N$ when $N$ is small and increases smoothly with $N$ when $N$ is large. Notice that the curve has some fluctuations when $N$ is large. This is because $N$, $m$ and $c$ are all integers with the relation $N=mc$, thus optimal $c$ can not take every integer value. We also observe that when $N$ is small (e.g., $N\leq 100$), the optimal $m$ decreases with increasing $N$; when $N$ becomes large (e.g., $N>100$), the optimal $m$ increases with $N$.

VII System Service Rate

From Theorem 3 and Lemma 3, we know that the average system time may not be a good performance metric to evaluate the tradeoff between the job computing time and the blocking probability when $\operatorname{\mathbb{E}}[T_{\text{cl}}]$ is large. In this section, we provide the system service rate $\sigma$ as another performance metric. This metric is adopted in [16] to evaluate the average computing speed. We will analyze the optimal $c$ and $m$ that maximize $\sigma$ with different service times. For a system with $N$ workers and $c$ groups, the expression of the system service rate is

[TABLE]

VII-A Exponential Service Time

From (10), considering the job completion distribution with $m$-fold replication as $\operatorname{Exp}(m\mu)$ and $m=\frac{N}{c}$, the system service rate is

[TABLE]

The following theorem gives the optimal number of groups $c$.

Theorem 7**.**

For the blocking system with Poisson($\lambda$) arrivals and $\operatorname{Exp}(m\mu)$ computing time, the system service rate $\sigma$ reaches the maximum at $c=1$ (i.e., $m=N$).

Proof.

From (11), we know that $\sigma(1)=\frac{N\mu}{1+K}$. Assume that $c_{1}\geq 2$,

[TABLE]

First, we consider the left-hand side of the above formula. Since $c_{1}\geq 2$ we have $1+(Kc_{1})^{c_{1}}/c_{1}!<c_{1}+c_{1}(Kc_{1})^{c_{1}}/c_{1}!$. Second, we consider the right-hand side of $\frac{\sigma(c_{1})}{\sigma(1)}$. Since $j\leq c_{1}-1$, we have

[TABLE]

Then we have,

[TABLE]

Therefore, we have the result that the system service rate $\sigma$ reaches the maximum at $c=1$. ∎

VII-A1 Numerical Analysis

We evaluate (11) for different values of $\lambda\in\{0.01,1,10\}$. Since it is easy to know that $\sigma$ decreases with the increasing $\lambda$ according to (5) and (11). We plot the normalized $\sigma$ vs. $c$ in Fig. 9 to see the changes in the system service rate. We consider a system with $N=24$ workers and the service time for each worker follows the exponential with $\mu=0.1$.

The figure shows that the system service rate $\sigma$ always decreases with the increasing $c$ and reaches the maximum at $c=1$. This observation is consistent with the conclusion of Theorem 7. When comparing the curves $\lambda=0.01$ and $\lambda=1$, we know that $\sigma$ decreases relatively smoother when the arrival rate is larger. When the arrival rate is sufficiently large (e.g., $\lambda=10$), $\sigma$ decreases sharply.

VII-B Shifted-Exponential Service Time

For a blocking system with $N$ workers and $c$ groups, when the job completion distribution with $m$-fold replication is $\operatorname{S-Exp}(\Delta,m\mu)$, the expression of the system service rate is

[TABLE]

We know that the optimal $c$ that maximizes the system service rate $\sigma$ changes with different parameters. However, considering the complexity of (12), optimizing $\sigma$ under a general shifted exponential service time is difficult. In the following, we separately analyze two important system parameters: the shift parameter and the job arrival rate. From the analysis, we want to know how the optimal $c$ changes with different values of the system parameters.

VII-B1 Shift Parameter

We analyze two special $(\Delta,\mu)$ parameter regions: 1) $\Delta\ll\frac{1}{\mu}$, which makes $T_{\text{job}}$ exponentially distributed, and 2) $\Delta\gg\frac{1}{\mu}$, which makes $T_{\text{job}}$ equal to constant $\Delta$. Apparently, the first case shows that the random part $\operatorname{Exp}(\mu)$ is much larger than the constant part $\Delta$, and the second case is on the contrary.

In the first case, the job completion distribution with $m$-fold replication is $\operatorname{Exp}(m\mu)$. According to Theorem 7, the system service rate $\sigma$ reaches the maximum at $c=1$. That is, when the random part is much larger than the constant part, it is better to decrease the number of groups to achieve a larger system service time. Next, we analyze the second case where the computing time is approximately constant. Then $\operatorname{S-Exp}(\Delta,m\mu)$ is approximated by the constant $\Delta$. The system service rate is $\sigma(c)=\frac{1-P_{b}(c)}{\Delta}$. According to Theorem 1, $1-P_{b}(c)$ increases with $c$. Therefore, $\sigma$ increases with $c$ and reaches the maximum at $c=N$.

The above analysis implies that the optimal $c$ that maximizes the system service rate lies between $1$ and $N$ for a general value of the shift parameter $\Delta$. When the random part is larger, the optimal $c$ is smaller and we should concentrate the computing resource on a few jobs; when the constant part becomes larger, the optimal $c$ also becomes larger and we should spread the resource to more jobs.

VII-B2 Job Arrival Rate

The arrival rate is also an important parameter that affects the system service rate. According to the system structure, we may infer that the system service rate changes as follows. When $\lambda$ is small, few jobs arrive. Even with a large replication factor, the blocking system can serve almost all jobs. Thus, introducing more redundancy may improve the system performance. When $\lambda$ is large, the system has to handle many jobs concurrently. It is, thus, reasonable to decrease the replication factor to process more jobs in the system.

In Theorem 8, we verify the first scenario in which $\lambda$ is sufficiently small.

Theorem 8**.**

For the blocking system with Poisson($\lambda$) arrivals and $\operatorname{S-Exp}(\Delta,m\mu)$ computing time, the system service rate $\sigma$ always reaches the maximum at $c=1$ when the arrival rate $\lambda\leq\frac{N\mu}{(\Delta N\mu+1)^{2}}$.

Proof.

When the replication factor $m=N$, the number of groups $c=\frac{N}{m}=1$. From Eq.(12), we have $\sigma(1)=\frac{N\mu}{(\Delta N\mu+1)(1+K(\Delta N\mu+1))}$, where $K=\frac{\lambda}{N\mu}$. When $c=c^{\prime}\geq 2$, we have $\sigma(c^{\prime})=\frac{N\mu}{(\Delta N\mu+c^{\prime})}(1-P_{b})\leq\frac{N\mu}{(\Delta N\mu+c^{\prime})}$.

To satisfy $\sigma(c^{\prime})\leq\sigma(1)$, we need

[TABLE]

That is,

[TABLE]

Since $c^{\prime}\geq 2$, the right-hand side reaches the minimum at $c^{\prime}=2$. Therefore, $\sigma(c^{\prime})\leq\sigma(1)$ holds for any $\lambda\leq\frac{N\mu}{(\Delta N\mu+1)^{2}}$.

∎

Next, we consider the scenario where $\lambda$ is sufficiently large. Using a method similar to the proof of Theorem 8, we find the optimal $c$ in the following lemma.

Lemma 7**.**

When the arrival rate $\lambda>\frac{2N\mu}{(\Delta N\mu)^{2}-2}$ (where $\Delta N\mu>\sqrt{2}$), the system service rate $\sigma$ always reaches the maximum at $c\geq 2$ .

Proof.

From Eq.(12), we have $\sigma(1)=\frac{N\mu}{(\Delta N\mu+1)(1+K(\Delta N\mu+1))}$ and $\sigma(2)=\frac{N\mu(1+\alpha)}{(\Delta N\mu+2)(1+\alpha+\alpha^{2}/2)}$, where $K=\frac{\lambda}{N\mu}$ and $\alpha=K(\Delta N\mu+2)$.

For $\sigma(2)$, we have

[TABLE]

To satisfy $\sigma(2)>\sigma(1)$, we need

[TABLE]

Then we have,

[TABLE]

When $\Delta N\mu>\sqrt{2}$, we have $\lambda>\frac{2N\mu}{(\Delta N\mu)^{2}-2}$. Then it is clear that $\sigma(2)>\sigma(1)$ holds. ∎

Generally speaking, decreasing the number of groups and increasing the replication factor provides better performance when the arrival rate is low. When the arrival rate is high, it is better to increase the number of groups. However, when the arrival rate is sufficiently high, the number of groups and the replication factor do not affect the system service rate.

VII-B3 Numerical Analysis

We evaluate $\sigma$ to see how the system service rate changes with the number of groups $c$. We consider a system with $N=24$ workers, the job arrives following the Poisson distribution with the rate $\lambda=1$, and the service time for each worker follows the shifted exponential distribution. Since the replication factor $m=\frac{N}{c}$ (Since both $m$ and $c$ are integers, we have $c\in\{1,2,3,4,6,8,12,24\}$). We assume $\mu=1$.

In Fig. 10, we evaluate (12) for three different values of $\Delta\in\{0.1,1,10\}$. We normalize $\sigma$ to observe the changes. When $\Delta\ll\mu$, $\sigma$ reaches the maximum at $c=1$, which means introducing more redundancy provides a higher system service rate. When $\Delta\gg\mu$, $\sigma$ reaches the maximum at $c=24$, which means that reducing proper redundancy provides a higher system service rate. Otherwise, the optimal $c$ lies between $1$ and $24$ (the optimal $c=4$ in Fig. 10).

We evaluate $\sigma$ to see how the system service rate changes with the number of groups $c$. We consider a system with $N=24$ workers. Jobs arrivals follow the Poisson distribution, and the worker computing time follows the shifted exponential distribution with the shift $1$ and the rate $1$.

In Fig. 11, we evaluate (12) for three different values of $\lambda\in\{0.01,1,10\}$. We normalize $\sigma$ to observe the changes. When $\lambda$ is sufficiently small, $\sigma$ reaches the maximum at $c=1$. When $\lambda$ is sufficiently large, $\sigma$ reaches the maximum at $c=24$. Otherwise, the optimal $c$ lies between $1$ and $24$ (the optimal $c=4$ in Fig. 11). Therefore, we conclude that the optimal $c$ increases with $\lambda$.

VIII Conclusion and Future Directions

We consider an edge computing system with limited storage and computing resources in which the arrival jobs will be sent to the cloud when the system is busy. We addressed the problems concerning the number of groups that optimize the job computing time and the job blocking probability. We find it impossible to simultaneously minimize the job computing time and the job blocking probability for both exponential and shifted exponential service times. Therefore, we use the average system time to evaluate the tradeoff between the job computing time and the job blocking probability. We find that the optimal number of groups minimizing the average system time changes with the job arrival rate and cloud time parameters. We also analyze the average system time when the computing resources are unlimited. We show that we need to concentrate the computing resources on a few jobs for the exponential service time to achieve the minimum average system time. However, we should balance concentrating and spreading the computing resources for the shifted exponential service time.

When the cloud time is significant, the average system time cannot perform well as the tradeoff between the job computing time and the job blocking probability. Then, we introduce the system service rate as a new combined metric and a single system performance indicator. We find that the system should always concentrate the computing resources on a few jobs for the exponential service time to achieve the maximum system service rate. However, the optimal number of groups changes with the shift parameter and the job arrival rate for the shifted exponential service time. This work sets the stage for many problems of interest to be studied in the future. We briefly describe three directions of immediate interest.

VIII-1 Computing Time and Blocking Probability Tradeoff for M/G/c/k Queues

As we mentioned in Sec. IV-B, our edge computing system is generally modeled as an M/G/c/k queue, where the capacity of the queue $k>c$. When all servers are busy, the new arrival job will wait in the limited-length queue, and the job will be blocked when the queue gets fully occupied. In this model, when we concentrate the computing resources on a few jobs to obtain a smaller computing time, the queue’s capacity does not decrease linearly. Thus, compared to the M/G/c/c queue results, spreading resources may not be the best strategy considering the job blocking probability.

VIII-2 Computing Time and Blocking Probability Tradeoff for Other Service Time Models

We analyzed the most common service time models. Other distributions, e.g., Pareto, bimodal, and Weibull, are also interesting. The optimal number of groups may behave differently for light and heavy-tailed distributions since their computing cost vs. time tradeoffs are qualitatively different [27].

VIII-3 System Service Rate for Blocking Systems

System service rate is a good performance metric to evaluate the distributed system[15]. We may analyze this metric for a general blocking system in the future. The blocked job will be dropped rather than sent to the cloud in such a system. If the users require every computing result, it is similar to the system in Sec. VII. If not, the users will send the request to the system again, increasing the job arrival rate. Analyzing job computing time and blocking probability under such schemes is open.