Lightweight Robust Framework for Workload Scheduling in Clouds

TL;DR

This paper introduces a lightweight, resource-efficient framework for robust workload scheduling in cloud environments that maintains stability despite unreliability and potential attacks.

Contribution

It presents a novel objective function and scanning strategy that ensure cloud stability without heavy resource use, extending to decentralized scheduling.

Findings

Framework guarantees stability under certain conditions

Decentralized scheduling maintains robustness

Outperforms naive scanning strategies in stability

Abstract

Reliability, security and stability of cloud services without sacrificing too much resources have become a desired feature in the area of workload management in clouds. The paper proposes and evaluates a lightweight framework for scheduling a workload which part could be unreliable. This unreliability could be caused by various types of failures or attacks. Our framework for robust workload scheduling efficiently combines classic fault-tolerant and security tools, such as packet/job scanning, with workload scheduling, and it does not use any heavy resource-consuming tools, e.g., cryptography or non-linear optimization. More specifically, the framework uses a novel objective function to allocate jobs to servers and constantly decides which job to scan based on a formula associated with the objective function. We show how to set up the objective function and the corresponding scanning procedure to make the system provably stable, provided it satisfies a specific stability condition. As a result, we show that our framework assures cloud stability even if naive scanning-all and scanning-none strategies are not stable. We extend the framework to decentralized scheduling and evaluate it under several popular routing procedures.

Tables

Table 1. Table 1: Representation of Instances in Amazon EC2

Instance type	Memory (GB)	vCPU	Storage (GB)
Standard	15	8	1,690
High-Memory	17.1	6.5	420
High-CPU	7	20	1,690

Equations

$$\underset{N\in S}{\max }\sum _{j=0}^J{Z}_j(t)N_j.$$

$$E[V(Q(t+1))-V(Q(t))|Q(t)=q]<-ϵ,\text{if }q\notin {\mathcal{Q}}_0,$$

$$E[V(Q(t+1))|Q(t)=q]<\infty ,\text{if }q\in {\mathcal{Q}}_0,$$

$$P({\tau }_q<\infty )=1,\forall q\in T$$

$$E[V(Q(t+1))-V(Q(t))|Q(t)=q]<-ϵ$$

$$\begin{array}[]{rl}&E[V(Q(t+1))-V(Q(t))|Q(t)=q]\\ =&E[\sum_{j}[Z_{j}(Q(t+1))^{2}-Z_{j}^{2}]|Q(t)=q]\\ =&E[\sum_{j}[(Z_{j}+Z_{j}(A)-Z_{j}(Alg))^{2}-Z_{j}^{2}]|Q(t)=q]\\ =&E[\sum_{j}[(Z_{j}(A)-Z_{j}(Alg))^{2}+\\ &+2Z_{j}(A-Z_{j}(Alg))]|Q(t)=q]\\ \leq&K+2E[\sum_{j}[Z_{j}\cdot Z_{j}(A)]|Q(t)=q]+\\ &-2E[\sum_{j}[[Z_{j}\cdot Z_{j}(Alg)]|Q(t)=q]\ .\end{array}$$

$$\begin{array}[]{rl}K+2E[\sum_{j}[Z_{j}\cdot Z_{j}(A)]|Q(t)=q]+&\\ &-2E[\sum_{j}[[Z_{j}\cdot Z_{j}(Alg)]|Q(t)=q]<0\ .\end{array}$$

$$a_j=({\lambda }_j+{\kappa }_j)((1-{\alpha }_j)+{\alpha }_j(\frac{{\lambda }_j}{{\lambda }_j+{\kappa }_j}+E(\frac{1}{L}))).$$

$$\forall q\in {\mathcal{Q}}^{\prime }\exists N\in SN\cdot Z(q)\ge K+a\cdot Z(q).$$

$$\begin{array}[]{rl}&\frac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}(1-\frac{\kappa_{j}}{\lambda_{j}+\kappa_{j}}L_{i})+\frac{\kappa_{j}}{\lambda_{j}+\kappa_{j}}(1+\frac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}L_{i})=\\ =&1+\frac{\lambda_{j}\kappa_{j}}{(\lambda_{j}+\kappa_{j})^{2}}(-L_{i}+L_{i})=1\ .\end{array}$$

$$a_j=\sum _i{p}_{i,j}[({\lambda }_j+{\kappa }_j)(1-{\alpha }_{i,j})+{\alpha }_j({\lambda }_j+{\kappa }_j)(\frac{{\lambda }_j}{{\lambda }_j+{\kappa }_j}+\frac{1}{L_i})],$$

Full text

1 Introduction

Cloud computing [14] enables ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources. It is becoming more and more popular for businesses to access computing facilities without investing in IT infrastructure.  [1, 2, 3, 4]. Cloud users send in resource requests in an online manner and the cloud provider allocates the required resources for the required amount of time. What is important, the resource allocation is transparent to the users. The provider allocates resources based on the system dynamicity and current system load. We refer readers to surveys on cloud computing for cloud technology [8, 5, 24].

While there is a growth in the use of cloud services, many potential users are still reluctant to deploy their business in the cloud. Major concerns are its reliability, security and stability [18]. There are different reliability and security issues depending on different delivery models of cloud services, including Software as Service (SaS), Platform as Service (PaS) and Infrastructure as Service (IaS). In this work we focus on the IaS model. This technology makes the users and provider reside at different locations and virtually access the resources over the Internet, therefore any security concerns threatening the Internet also threaten the cloud. In particular, we consider the scenarios when part of the workload is unreliable, e.g., fault-prone or generated by malicious sources, and propose a lightweight framework that combines load management and detection of unreliable traffic. We investigate how to strike a balance between efficient workload scheduling and packet/job scanning so that we can maintain stability (as a guarantee of bounded buffers at machines) without sacrificing too much resources to filter out the unreliable part of the workload. IaS provides users with computing infrastructure in the form of Virtual Machines (VM). Following [11], we assume that the users request resources such as memory, CPU and storage, for a certain amount of time in the form of VMs; this corresponds to a job to be done. Upon receiving the requests (typically in a form of packets), the system has to allocate the required resources by scheduling the VMs on the server. We extend the model in [11] by considering scenarios where part of the workload is genuine and the other unreliable. Genuine traffic comes from real users; completing these requests counts towards system’s work done. Unreliable traffic is subject to failures or comes from attackers, who aim to disrupt the system by issuing requests that occupy resources; completing these does not count as proper work done. We adopt a classic reliability and security tool of packet scanning to detect these malicious packets [15]. While scanning is able to distinguish genuine from unreliable requests,it consumes and wastes resources that would normally be used for serving genuine workload.

On the other hand, as we do not know whether the packets are faulty/fake until we scan them, we may also waste time and resources in scanning genuine packets. Therefore, the scheduling algorithm needs to strike a balance between the resources wasted by scanning and by performing unreliable requests without scanning them.

We consider centralized and distributed scheduling algorithms. In the centralized setting, there are central queues, and upon arrival jobs are added to the central queues corresponding to the requested type of VMs; recall that there is a limited number of types of VMs (as each of VMs is in fact a small operating system [11]) and each job is allocated to a VM of the requested type. When the resources become available, the centralized scheduling algorithm determines which set of jobs is to be served and to which servers the VMs are mapped to. In the distributed setting, each server has its own queues; upon arrival, a job request is forwarded to some server and stored in the server’s local queue corresponding to the requested type of VMs. The requests from these local queues are served by a distributed scheduling algorithm locally on the server.

The system is stable if the queues do not tend to increase without bound. We aim to characterize the maximum arrival rates of genuine and unreliable requests under which there is an algorithm to maintain the stability of the system and to develop such algorithm if it exists. In addition, to guarantee quality of service, we also measure job latency, which is defined as the amount of time a job resides in the system since its arrival. We present the precise model in Section 2 and the proposed algorithms and analysis in Sections 3 to 6. We present the experiments and conclude in Sections 7 and 8.

1.1 Related Work

Apart from maintaining stability, there are many other design issues related to workload management in cloud computing. Cloud utilization has been considered in [22, 16, 23]. Optimizing other costs of running the services has been considered [17, 21, 22].

The algorithms we propose here are inspired by the MaxWeight algorithm analyzed in [20] in the context of scheduling genuine workload only, and could be seen as its efficient generalizations to unreliable environments. The MaxWeight algorithm has been since investigated extensively [13, 19, 12]. Detecting and distinguishing unreliable or malicious from genuine requests and a number of approaches have been proposed [15, 7]. In this paper, we assume that such a tool to scan a packet and detect potentially unreliable or malicious packages is available. The authors in [10] studied jobs with unknown duration and analyzed several decentralized approaches and showed that some are throughput-optimal while others are not. Another study [9] aimed to optimize recovery time after failures; it is different from our aim to prevent the impact of failures by tailored combination of scheduling and scanning tools.

1.2 Our Contributions

We propose a lightweight robust framework to manage workload in clouds under unreliable workload scenarios. Extending the model in [11, 20], we propose to detect unreliable part of the traffic by scanning only some specifically selected jobs without sacrificing too much resources.

• •

We propose a theoretical model to capture the essence of this conditional scanning and show that under a certain system capacity region and stochastic arrival pattern of genuine and unreliable jobs there exists an algorithm, called RobustMaxWork, that manages the workload while maintaining queue stability (i.e., the queue is bounded and does not grow to infinity size).

• •

We show how to efficiently compute the optimal scanning strategy (vector).

• •

We prove that there is no stable algorithm for the workload outside the capacity region for which RobustMaxWork is stable.

• •

We propose several distributed versions of RobustMaxWork and discuss various extensions of theoretical results.

• •

We evaluate the algorithms and the proposed model using extensive simulations, with respect to the maximum and average latency over time.

2 Model

We consider a cloud system modeled by a network of physical machines that have limited available resources (for instance, CPU, memory, storage, …) and is supposed to be able to process an ongoing stream of jobs.

Servers. We consider a set of $n$ networked servers (physical machines). Each server has its own resources that it can distribute among jobs, that is, for each resource it has a fixed capacity.

Jobs. A job is specified by its type and length. Since there are limited number of virtual machine types, we only consider limited number of job types – there are $J$ types of jobs. Each type is a set of demands for resources; more specifically, for each available server resource a job type has a number specifying how much of this resource is required in order to process any job of this type.

There are $I$ different lengths of jobs possible: $L_{1},\ldots,L_{I}$

We consider online random arrival model, where new jobs arrive independently of each other and are identically distributed across all time slots, and the variance of arrival length is finite. Let $\lambda_{i,j}$ denote expected sum of lengths of genuine (i.e., user-generated) type-$j$ jobs of length $L_{i}$ that arrive per time slot, for any positive integers $j\leq J$ and $i\leq I$.

Processing jobs and feasible configurations. Each server can process a set of jobs simultaneously, as long as the cumulative amount of each resource used by these jobs does not exceed the server capacity for this resource. Processing jobs is done in synchronous time steps, also called rounds. The whole system capacity is a linear sum of capacities of all the servers. Given job types and server capacities, one can compute the set $\mathcal{S}$ of all feasible configurations, where feasible configuration denotes a vector $N=(N_{1},\ldots,N_{J})$ such that the system can process simultaneously $N_{j}$ type-$j$ jobs, for every $j$.

Malicious jobs and security tools. Let $\kappa_{i,j}$ denote the expected sum of lengths of malicious jobs of type-$j$ of length $L_{i}$ that arrive per time slot. Similarly as genuine jobs, malicious jobs arrive independently of each other and are identically distributed across all time slots, and the variance is finite. We assume that we have a scanning tool that, given a job, can detect whether it is a genuine user request (we will call it a good job) or a malicious request (a malicious job). Each scanning takes $1$ time slot per job and requires same resources as the original job (scanning is done on the same virtual machine).

Central scheduler. We consider a central scheduler with a queue of all injected, but not yet finished, jobs. The scheduler decides which servers process which jobs for the next time slot. After this time slot, all unfinished jobs return to the scheduler with saved progress and can be processed further at a later time and by a different server. This property of a system is called preemptiveness. The centralized algorithm, called RobustMaxWork, will be introduced in Section 3.

Distributed scheduler. In decentralized approach all servers maintain separate queues for jobs of type $j$, therefore when a job arrives decision has to be made as to which server to route the job to. Each server runs locally a protocol RobustMaxWork with respect to its local queues. The distributed implementations of algorithm RobustMaxWork will be presented in Section 6.

Notation.

• •

$n$ denotes the number of servers in the cloud;

• •

$I$ denotes the number of different job lengths;

• •

$J$ denotes the number of different job types;

• •

$A(t)=(A_{1}(t),\ldots,A_{J}(t))$ denotes the vector of sets of type-$j$ jobs, for $j\leq J$, which arrive to the system in the beginning of time slot $t$;

• •

$Q(t)$ denotes the vector of queue lengths (i.e., sum of lengths of jobs in the queue) for each type of jobs in the beginning of time slot $t$;

• •

$Q_{j}(t)$ denotes the total length of users’ and malicious type-$j$ jobs, for $j\leq J$, in the beginning of time slot $t$;

In addition there are the following notations regarding RobustMaxWork algorithm:

• •

$\alpha_{i,j}$ is a probability of scanning type-$j$ job of length $L_{i}$; the algorithm may implement a specific scanning strategy, i.e.,

use a specific vector $\alpha$.

• •

$X_{j}(t)$ is the total length of queued type-$j$ jobs that will not be scanned, taken in the beginning of time slot $t$ (i.e., the algorithm scanned them already or decided not to scan them at all);

• •

$Y_{j}(t)$ is the total length of queued type-$j$ jobs that will be scanned, taken in the beginning of time slot $t$ (i.e., the algorithm has already decided to scan them, but has not scanned them yet);

• •

$Z_{j}(t)=Z_{j}(Q(t))$ is the expected time required to process type-$j$ jobs stored in queue in the beginning of time slot $t$; the formula with an explanation for it will be given in section 3;

• •

$a_{j}$ is the expected time required to process type-$j$ jobs that arrive in one time slot; the formula with an explanation for it will be given in section 5;

Whenever time slot $t$ is clearly fixed or understood from the context, we may omit an argument $t$ from the formulas.

Scanning strategies. We will compare the following scanning strategies:

• •

Scan-None— always executes a job without scanning, i.e., $\alpha_{i,j}=0$ for all $i,j$;

• •

Scan-All — scans all jobs except those with processing time shorter or equal to the scanning time (recall that scanning takes $1$ time slot), i.e., $\alpha_{i,j}=0$ for $L_{i}\leq 1$ and $\alpha_{i,j}=1$ otherwise;

• •

Scan-Opt— will be defined in section 5.2.

Stability. We say that, given arrival rates $\lambda$ and $\kappa$, the algorithm is stable if the expected queue size at any fixed time is bounded, i.e. $\limsup\limits_{t\rightarrow\infty}E[\sum_{j}Q_{j}(t)]<\infty$.

3 Main Algorithm — Centralized Version

Algorithm RobustMaxWork (see Algorithm 25, with Algorithms 14, 3 and 8 as sub-procedures) is parametrized by: scanning vector $\alpha=(\alpha_{i,j})_{i\leq I,j\leq J}\in[0,1]^{I\times J}$, vector of rates of genuine user’s requests $\lambda=(\lambda_{i,j})_{i\leq I,j\leq J}$, and vector of rates of malicious requests $\kappa=(\kappa_{i,j})_{i\leq I,j\leq J}$ . Upon arrival of type-$j$ job of length $L_{i}$, algorithm RobustMaxWork decides to scan it with probability $\alpha_{i,j}$ (c.f., the first for all loop in Algorithm 25).

The key idea of RobustMaxWork is to measure the expected time required to process all jobs of each type $j$ and prioritize the jobs of type which accumulated the most. The expected time (also called expected work) required to process all jobs of type $j$ accumulated in queue at time $t$ is denoted by $Z_{j}(t)$.

It takes $X_{j}$ time to process jobs that will not be scanned (jobs contributing to $X_{j}$). Jobs contributing to $Y_{j}$ will need to be scanned (by definition of $Y_{j}$), which requires $Y_{j}\cdot E(1/l_{j})$ expected time. In expectance $\lambda_{j}/(\lambda_{j}+\kappa_{j})$ fraction of scanned jobs are genuine, so after scanning, they still must be processed, taking in total $Y_{j}\cdot\lambda_{j}/(\lambda_{j}+\kappa_{j})$ time. $\kappa_{j}/(\lambda_{j}+\kappa_{j})$ fraction of scanned jobs are fake and after scanning they take no more processing time. Therefore:

$Z_{j}(t)=X_{j}(t)+Y_{j}(t)\cdot(\lambda_{j}/(\lambda_{j}+\kappa_{j})+E(1/\ell_{j}))$,

where $\ell_{j}$ is a (random) length of arriving type-$j$ jobs.

The algorithm then computes values $Z_{j}$ (c.f., the second for all loop in Algorithm 25) and finds configuration $N$ from the set of feasible server configurations $\mathcal{S}$ that maximizes the sum $\sum_{j=0}^{J}Z_{j}(t)N_{j}$, i.e., the objective of the algorithm in each time slot $t$ is:

[TABLE]

This configuration is denoted by $N^{\prime}$. The quick intuition behind this function is that the more jobs of a given type accumulate, the more weight should be put to scheduling the jobs of that type in order to prevent further accumulation. $Z_{j}$ here is the weight given to jobs of type $j$.

Finally, in the last for all loop, the algorithm processes $N^{\prime}_{j}$ jobs of type $j$, for each $j\in\{1,\dots,J\}$; that is, from each processed job it executes a unit of it and the total size of $Q_{j}$ decreases by $N^{\prime}_{j}$ at the end of time slot $t$. It is done by calling procedure Process_job($j$), c.f., Algorithm 14. If $N^{\prime}_{j}$ is larger than the number of different type-$j$ jobs in the queues, RobustMaxWork processes as many type-$j$ jobs as possible instead, each time processing a unit of each such job. (c.f., the second part of procedure Process_job($j$)). If $N^{\prime}_{j}$ is smaller than the number of different type-$j$ jobs in the queues, RobustMaxWork has to decide which type-$j$ jobs to process (c.f., the first part of procedure Process_job($j$)). It repeats $N^{\prime}_{j}$ times:

• •

with probability $X_{j}/(X_{j}+Y_{j})$ it processes a job that will not be scanned (i.e., a job that contributes to $X_{j}$),

• •

with probability $Y_{j}/(X_{j}+Y_{j})$ it scans a job pending for scanning (i.e., a job that contributes to $Y_{j}$).

If there are not enough jobs contributing to $X_{j}$, it processes all jobs contributing to $X_{j}$ and as many jobs contributing to $Y_{j}$ as possible, so that altogether it processes $N_{j}$ type-$j$ jobs. Vice versa, if there are not enough jobs contributing to $Y_{j}$, it processes all jobs contributing to $Y_{j}$ and as many jobs contributing to $X_{j}$ as possible. Processing and/or scanning a specific type-$j$ job is done by calling sub-procedures Process_job_X($j$) and/or Process_job_Y($j$), respectively (c.f., Algorithms 3 and 8, resp.) directly from the execution of procedure Process_job($j$).

In short words, we could describe a single time slot of an execution of algorithm RobustMaxWork as follows. We always have $X_{j}+Y_{j}=Q_{j}$, for any type-$j$, as each job is either waiting for scanning or not (i.e., has been already scanned or is not selected for scanning at all). Whenever a type-$j$ job of length $L_{i}$ arrives, with probability $\alpha_{i,j}$ its length is added to $Y_{j}$, otherwise its length is added to $X_{j}$. When the algorithm executes one unit of a job contributing to $X_{j}$, $X_{j}$ is reduced by $1$. When the algorithm executes one unit of job contributing to $Y_{j}$, it means it scans it — if it was a genuine user job, its length is removed from $Y_{j}$ and added to $X_{j}$ (so the algorithm spent one round on scanning, but the sum $X_{j}+Y_{j}$ remains the same); if it was a malicious job, its length is removed from $Y_{j}$.

4 Analysis

In this section we prove that algorithm RobustMaxWork is stable if there is $\varepsilon>0$ and vector $a$ such that $a\in(1-\varepsilon)\cdot co(S)$.

Theorem 1**.**

The RobustMaxWork algorithm is stable for all arrival patterns $\lambda,\kappa$, for which there exists a stable algorithm.

In the remainder of this section we will prove Theorem 1. We will need the following result (extension of Foster’s criteria for irreducible Markov chains).

Theorem 2** ([6]).**

Consider a Markov chain $Q(t)$ with state space $\mathcal{Q}$. Consider a random walk on it, starting from a state $x$. Let $\tau_{x}$ denote the time when the random walk first reaches some recurrent state (or infinity if it never reaches any). If there exists a lower bounded real function $V:\mathcal{Q}\rightarrow\mathbb{R}$, an $\epsilon>0$ and a finite subset $\mathcal{Q}_{0}$ of $\mathcal{Q}$ such that

[TABLE]

[TABLE]

then we have

[TABLE]

and all states $q\in\cup_{j=1}^{\infty}R_{j}$ are positive recurrent.

Let $V(Q(t))=\sum_{j}(Z_{j}(Q(t)))^{2}$. Note that $V(Q(t))\geq 0$ for all possible queue states $Q(t)\in\mathcal{Q}$. We show that there exist two positive numbers $b,\epsilon$ such that the inequality

[TABLE]

holds for all $q\in\mathcal{Q}$ for which $q_{j}\geq b$.

Let $A(t)$ denote the vector of arrival lengths for each type of job, with distinction between jobs that will be scanned and jobs that will not be scanned, in the beginning of time slot $t$. Let $Alg(t)$ denote the vector of queue changes due to algorithm decisions for each type of job, with distinction between jobs that will be scanned and jobs that will not be scanned, in the beginning of time slot $t$. We will be using $Z_{j}$ as a shorthand of $Z_{j}(Q(t))$, $A$ as a shorthand of $A(t+1)$, and $Alg$ as a shorthand for $Alg(t+1)$.

[TABLE]

The last inequality comes from $E[\sum_{j}[(Z_{j}(A)-Z_{j}(Alg))^{2}]|Q(t)=q]$ being upper bounded under assumption that the variances of arrival lengths are finite;

we denoted this upper bound by $K$.

Lemma 1**.**

There exists finite set $\mathcal{F}\subseteq\mathcal{Q}$ such that for all $q\in\mathcal{Q}-\mathcal{F}$:

[TABLE]

To prove Lemma 1, we need the following result:

Lemma 2**.**

For almost all queue states $q$ there exists a feasible configuration $N\in S$ such that $N\cdot Z\geq K+a\cdot Z$ (where $Z=Z(q)$ and $\cdot$ is scalar product).

Proof.

We remind that vector $\alpha$ is chosen in such a way that the vector $(1+\epsilon)a$ lies inside convex hull of set $\mathcal{S}$, further denoted by $co(S)$ (set $\mathcal{S}$ is the set of feasible server configurations $N$), where

[TABLE]

Let $a^{\prime}$ be a vector (a point) corresponding to intersection of vector $a$ with a face $F$ of $co(S)$. $a^{\prime}\geq(1+\epsilon)a$ is a linear combination of some feasible configurations $N^{(1)},\dots,N^{(k)}$ (configurations on the face $F$). Therefore for any non-negative vector $Z$ there exists a configuration $N$ such that $N\cdot Z\geq a^{\prime}\cdot Z$ (at least one of $N^{(1)},\dots,N^{(k)}$ is such a configuration). So $N\cdot Z\geq(1+\epsilon)a\cdot Z$, thus $N\cdot Z-\frac{\epsilon}{1+\epsilon}N\cdot Z\geq a\cdot Z$.

Consider the set of queue states $\mathcal{Q}^{\prime}=\{q:\exists N\ \frac{\epsilon}{1+\epsilon}N\cdot Z(q)\geq K\}$.

[TABLE]

In order to complete the proof of the lemma, it remains to show that $\mathcal{Q}-\mathcal{Q}^{\prime}$ is finite.

Note that $Z_{j}$ is a monotonically increasing function and if direction of $q$ is same as $q^{\prime}$ then direction of $Z(q)$ is same as $Z(q^{\prime})$. For each possible direction of $Z(q)$, for all $q\in\mathcal{Q}$ different from $\overrightarrow{0}$, there exists $q^{\prime}\in\mathcal{Q}^{\prime}$ such that $Z(q^{\prime})$ is in the same direction as $Z(q)$ ($q^{\prime}$ may be some multiplicity of $q$ that is large enough to “kill” constant $K$). Then for any $q^{\prime\prime}\geq q^{\prime}$ we have $Z(q^{\prime\prime})\geq K+b\cdot Z(q^{\prime\prime})$.

For each possible direction of $Z$ we can take the minimum vector $q_{0}\in\mathcal{Q}^{\prime}$ such that $Z(q_{0})$ is in the considered direction. Then we take one vector $q_{0}^{\prime}$ that is greater than all $q_{0}$’s for each direction.

In each direction there is a finite number of vectors smaller than the corresponding $q_{0}$. The lengths of vectors $q_{0}$ are bounded, as the optimized function is continuous and considered on a compact set of directions.

Hence the number of configurations of smaller length than the supremum $q_{0}$ is finite, and so $\mathcal{Q}-\mathcal{Q}^{\prime}$ is finite.

∎

Proof of Lemma 1.

Note that in Lemma 1, $E[\sum_{j}[Z_{j}\cdot Z_{j}(A)]]$ is the expected value of the scalar product $Z\cdot Z(A)$ and $E[\sum_{j}[[Z_{j}\cdot Z_{j}(Alg)]$ is the expected value of the scalar product $Z\cdot Z(Alg)$. According to Lemma 2, inequality from lemma 1 is true for almost all queue states, i.e., there exists a finite set of states $\mathcal{F}$ such that for all $q\in\mathcal{Q}-\mathcal{F}$ the desired inequality holds. ∎

Proof of Theorem 1.

According to Lemma 1 and Theorem 2, given arrival rates for which there exists some stable algorithm, RobustMaxWork algorithm reaches positive recurrent state in a finite time, therefore it is stable. ∎

5 Determining feasible capacity region and scanning frequency

5.1 Feasible capacity region

Lemma 3**.**

Processing a type-$j$ job for 1 time slot decreases $Z_{j}(Q(t))$ by 1 on average.

Proof.

If the processed step was not scanning (i.e., processing a job from $X_{j}$) then trivially $Z_{j}$ decreased by 1.

If the processed step was scanning a job of length $L_{i}$ (i.e., processing a job from $Y_{j}$), then:

• •

before scanning that job contributed $\frac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}\cdot L_{i}+1$ weight towards $Z_{j}$;

• •

with probability $\frac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}$ it was a genuine job, so after scanning it contributes $L_{i}$ towards $Z_{j}$ (increase in weight);

• •

with probability $\frac{\kappa_{j}}{\lambda_{j}+\kappa_{j}}$ it was a malicious job, so after scanning it contributes [math] towards $Z_{j}$ (decrease in weight).

Therefore, on average $Z_{j}$ decreases by

[TABLE]

∎

Recall that $\alpha_{i,j}$ is the probability of scanning type-$j$ job of length $L_{i}$, and $A(t)$ denotes the vector of arrival lengths for each type of job, with distinction between jobs that will be scanned and jobs that will not be scanned in the beginning of time slot $t$.

Let $a_{j}=a_{j}(\alpha,\lambda,\kappa)=E[Z_{j}(A(t))]$ be the expected weight of type-$j$ jobs that arrive per time slot (arrivals are i.i.d. across time slots, so $E[A_{j}(t)]=E[A_{j}(t+1)]$ for all $t$). Then

[TABLE]

where $p_{i,j}$ is the probability that type-$j$ job has length $L_{i}$. Addend ($\lambda_{j}+\kappa_{j})(1-\alpha_{i,j})$ corresponds to the weight of good and malicious jobs that will not be scanned (so it contributes to $X_{j}$). Addend $\alpha_{j}(\lambda_{j}+\kappa_{j})\cdot\frac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}$ corresponds to the weight of good jobs that will be scanned but without scanning taken into account yet (so it contributes to $Y_{j}$). Addend $\alpha_{j}(\lambda_{j}+\kappa_{j})\cdot 1/L_{i}$ corresponds to the weight of scanning good and malicious jobs (so it also contributes to $Y_{j}$). Let $a=(a_{1},\dots,a_{J})$.

Theorem 3**.**

If arrivals $\lambda$ and $\kappa$ are such that for all vectors $\alpha$ arrivals $a\notin co(\mathcal{S})$ then no algorithm is stable.

Proof.

Consider arrival rates $\lambda$, $\kappa$ and scanning probabilities $\alpha$ such that $a\notin co(\mathcal{S})$.

We will show that for every algorithm there exists $j$ such that $E[Z_{j}(Q(t))]$ is unbounded.

In every time slot $t$ the weight of queues $Z(Q(t))$ is changing on average by $E[Z(Q(t+1))-Z(Q(t))]=E[Z(Q(t)+A(t+1)-Alg(t+1))-Z(Q(t))]=E[Z(A(t+1))-Z(Alg(t+1))]=a-E[Z(Alg(t+1))]$.

Note that $a\notin co(S)$, while $E[Z(Alg(t+1)]=N(t+1)\in co(S)$ (according to Lemma 3). If we consider multiple time slots and any combination of algorithm decisions $N(t)$ then the vector of weights of queues $Z(Q(t))$ is growing in the direction of vector $a$. Therefore there exists $j$ such that $Z_{j}(Q(t))$ is unbounded with regard to $t$. Therefore $Q_{j}(t)$ is unbounded with regard to $t$, which is contradictory with our definition of stability. ∎

5.2 Optimal scanning frequencies

Theorem 4**.**

If there exists a vector of scanning frequencies $\alpha^{(0)}$ such that given job arrival rates $\lambda$ and $\kappa$ are inside the capacity region (i.e., $(1+\epsilon)a(\alpha^{(0)},\lambda,\kappa)\in co(S)$ as defined in section 5), then there exists a vector of scanning frequencies $\alpha^{(1)}\in\{0,1\}^{I\times J}$ such that these job arrivals are inside the capacity region ($(1+\epsilon)a(\alpha^{(1)},\lambda,\kappa)\in co(S)$).

Proof.

Recall the following properties that we will be using in the proof. First, scanning uses same resources as the scanned job. Second, if vector $N\in S$, then also $N^{\prime}\in S$ for all $0\leq N^{\prime}\leq N$. Therefore, if $a\in co(S)$ then $a^{\prime}\in co(S)$ for all $0\leq a^{\prime}\leq a$. (Inequalities between vectors $a\leq b$ mean that for all $i$, $a_{i}\leq b_{i}$.)

Assume vector $a(\alpha^{(0)},\lambda,\kappa)\in co(S)$, where $a_{j}(\alpha,\lambda,\kappa)=\sum_{i}p_{i}[(\lambda_{j}+\kappa_{j})(1-\alpha_{i,j})+\alpha_{i,j}(\lambda_{j}+\kappa_{j})(\dfrac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}+1/L_{i})))]$, as defined in section 5. Therefore, function $a_{j}(\alpha,\lambda,\kappa)$ is independent of $\lambda_{k},\kappa_{k},\alpha_{i,k}$ for $k\neq j$ and for all $i$. Given fixed $\lambda$ and $\kappa$, $a_{j}(\alpha,\lambda,\kappa)$ is a linear combination of scanning frequencies $\alpha_{i,j}\in[0,1]$, for all $i$.

Therefore the vector $\alpha_{j}=(\alpha_{1,j},\alpha_{2,j},\dots,\alpha_{I,j})$ that minimizes $a_{j}$ is one of the extreme points of region $[0,1]^{I}$. Furthermore, we can easily compute $\alpha_{i,j}^{(1)}$ for all $i$ that minimize $a_{j}$, since each summand $p_{i}[(\lambda_{j}+\kappa_{j})(1-\alpha_{i,j})+\alpha_{i,j}(\lambda_{j}+\kappa_{j})(\dfrac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}+1/L_{i})))]$ is independent of all other summands. Value $\alpha_{i,j}^{(1)}$ that minimizes this summand is [math], if $1\leq\dfrac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}+1/L_{i}$, and $1$ if $1>\dfrac{\lambda_{j}}{\lambda_{j}+\kappa_{j}}+1/L_{i}$, for each $i,j$, and can be computed using Algorithm 7.

This gives $a(\alpha^{(0)},\lambda,\kappa)\geq a(\alpha^{(1)},\lambda,\kappa))$, which means that $(1+\epsilon)a(\alpha^{(1)},\lambda,\kappa))\in co(S)$. ∎

6 Decentralization

Centralized scheduler uses the same queue for all type-$j$ jobs that are waiting in the system. In decentralized approach each server maintains its own queues for jobs of type-$j$, therefore when a job arrives a decision has to be made as to which server to route the job. In this section we specify and analyze six different decentralized implementations of the main algorithm RobustMaxWork from Section 3with different routing procedures.

Another distinct property of decentralized approach is using the main RobustMaxWork algorithm at each server to make scheduling decision.

In [10] the authors introduced the notion of refresh time. A time slot $t$ is a global refresh time if there is no job currently in queues in all the servers at the beginning of $t$. Local refresh time occurs when there is no job currently in service at the beginning of time slot $t$ in a server $v$. Note that a global refresh time happens when all servers have their local refresh times. In practice global refresh times occurs rarely and intuitively it happens to be more rear as the number of servers increase. Due to this phenomenon, all our decentralized algorithms will rely only on the local refresh times at servers.

Decentralized RobustMaxWork for each server $v$. Upon local refresh time, server $v$ keeps idle until some of its queues are non-empty. Then it applies RobustMaxWork with respect of its queues to find configuration $N^{\prime}$. Then it keeps scheduling jobs according to this configuration every round until the next local refresh time (i.e., there is no suitable job to apply configuration $N^{\prime}$. This scheme is repeated.

Below we describe six specification of the decentralized RobustMaxWork with different routing policies. Algorithm 1 and 3 were analysed in  [10], 4 was analysed in  [11], and we designed 2, 5, and 6 in this work. We will discuss the performance of the new algorithms and their comparisons with the existing ones in the results section.

Algorithm 1: RobustMaxWork_JSQ. Joint Shortest Queue (JSQ) paradigm is used to route a newly arrived job, that is, it is sent to the server with the queue with the smallest number of jobs of type-$j$, where $j$ denotes the type of the arrived job. This algorithm was analyzed in the context of cloud workload in [10].

Algorithm 2: RobustMaxWork_JSW. Joint Shortest Work (JSW) is used for routing a newly arrived job; i.e., it is sent to the server with the minimum work load of type-$j$, where the work load is defined as a sum of lengths of jobs stored in the local queue of type-$j$.

Algorithm 3: RobustMaxWork_UR. Uniformly Random (UR) routing is used for forwarding newly arrived job: each job that arrives into the system is routed to one of the servers chosen uniformly at random. This algorithm was analyzed in the context of cloud workload in  [10]

Algorithm 4: RobustMaxWork_RR. Round Robin (RR) routine is used for allocating newly arrived jobs to the servers: for each type-$j$ there is a pointer going cyclically along the servers, showing which was the last server to which a type-$j$ job was allocated; when a new type-$j$ job arrives it is sent to the next server (modulo $n$) and the pointer is advanced to that server.

Algorithm 5: RobustMaxWork_P2Q. Power of two Choices combined with selection of the Shortest Queue (P2Q) is used for routing a newly arrived job of a type-$j$: two servers are sampled uniformly at random, and the job is routed to the server with the shorter type-$j$ queue.

Algorithm 6: RobustMaxWork_P2W. Power of two Choices combined with selection of the Shortest Work load (P2W) is used for routing a newly arrived job of a type-$j$: two servers are sampled uniformly at random, and the job is routed to the server with the smaller workload of type-$j$(i.e., where the total length of type-$j$ jobs in the local queue is shorter).

7 Simulations

7.1 Experiment Setting

The setup for simulations, described in this section, is based on the one in Maguluri et al. [11].

Servers and VMs

We consider a server with 30 GB memory, 30 EC2 computing units and 4000G storage space. There are $100$ identical servers in the cloud. Arriving jobs are served in the cloud based on three types of virtual machines described in Table 1. This gives three maximal configurations available at each server: $(2,0,0)$, $(1,0,1)$ and $(0,1,1)$.

Job arrivals

We use the generic arrival vector $\lambda^{*}=0.99\cdot(1,1/3,2/3)$ for the genuine users’ workload, which is located at the border of the server capacity area (it is easy to observe that it is a normalized linear combination of the three maximal configurations, additionally re-scaled by factor $0.99$).

In each time step a job of type $j=1,2,3$ is selected with probability $\frac{\lambda^{*}_{j}}{130.5}$, and its length is chosen according to the length distribution described below with the mean length $130.5$.

Similarly as above,

we define a malicious workload using a generic arrival vector $\kappa^{*}=(0.7,0.01,0.01)$, and the procedure of generating a malicious traffic is analogous as above for generating the genuine users’ one.

Note that each of the arrival rates $\lambda^{*}$ and $\kappa^{*}$ is within the capacity range of a server, whereas the combined workflow rate $\lambda^{*}+\kappa^{*}$ is not.

Job size distribution

When a new job is generated, with the probability of $0.7$ it is an integer that is uniformly distributed in the interval $[1,50]$, with the probability of $0.15$ it is an integer uniformly distributed in the interval $[251,300]$, and with the probability of $0.15$ it is an integer uniformly distributed in the interval $[451,500]$.

Note that there are $150$ possible job lengths, and the mean length is $130.5$, as assumed in the definition of arrival rates.

Set up of simulations.

Since there are $100$ homogenous servers, the overall arrival rates are: $\lambda=100\cdot\lambda^{*}=(99,33,66)$ for genuine workload, and $\kappa=100\cdot\kappa^{*}=(70,1,1)$ for malicious workload. The job size distribution is as specified above, same for each job type. We computed the following optimal scanning vector $\alpha^{*}$ for this setting, more precisely, the vector minimizing expected arriving weight:

• •

$\alpha^{*}_{i,1}=0$ for $L_{i}\leq 2$,

• •

$\alpha^{*}_{i,2}=0$ for $L_{i}\leq 34$,

• •

$\alpha^{*}_{i,3}=0$ for $L_{i}\leq 50$,

• •

$\alpha^{*}_{i,j}=1$ otherwise.

Each execution includes $4,000,000$ time steps. We monitor the following parameters every $200,000$ time steps:

• •

Average queue size by the recorded time step;

• •

Maximum queue size by the recorded time step;

• •

Average latency by the recorded time step;

• •

Maximum latency by the recorded time step.

In the first part, we output the results of the above measurements for centralized protocols:

LambdaFlow:

RobustMaxWork applied for genuine flow only, (i.e., only with genuine arrival rate $\lambda$), and no scanning is applied (i.e., $\vec{\alpha}=\mathbf{0}$);

ScanOPT:

RobustMaxWork applied for simultaneous genuine and malicious flows, with scanning defined by vector $\alpha^{*}$;

ScanALL:

RobustMaxWork applied for simultaneous genuine and malicious flows, with scanning all jobs of size bigger than $1$ (i.e., for every $j=1,2,3$, $\vec{\alpha_{1,j}}=0$ and $\vec{\alpha_{i,j}}=1$ for every $i>1$);

ScanNONE:

RobustMaxWork applied for simultaneous genuine and malicious flows, with no scanning (i.e., $\vec{\alpha}=\mathbf{0}$).

As by theoretical part, it is expected that the first two executions should be stable while the last one is not. We expect that the third execution is also not stable, which would justify our research quest for searching of suitable scanning vector. In order to visualize it, we also display differences and ratios between the second and the third executions — the stable and the potentially unstable one.

The second part of simulations is dedicated to decentralized algorithms. We study how different routing protocols influence stability, when applied to the RobustMaxWork with the optimally selected scanning vector $\alpha^{*}$. We compare the six decentralized implementations with the centralized one, RobustMaxWork-OPT. They are denoted by ScanOPT_JSQ, ScanOPT_JSW, ScanOPT_UR, ScanOPT_RR, ScanOPT_P2Q, ScanOPT_P2W, and ScanOPT, respectively.

7.2 Results

7.2.1 Centralized Approaches

In order to study throughput-optimality of the scanning strategies, we recorded the latency over time for the different scanning strategies used, i.e., ScanALL, ScanOPT and ScanNONE, comparing them with the execution LambdaFlow of the genuine workload only. In Figure 1, the average latency of the ScanNONE and ScanALL strategies grow rapidly, while it stabilizes for the ScanOPT strategy. The right part of the Figure is the zoomed left part, in order to see clearly the performance of ScanOPT versus ScanALL. The performance of ScanALL strategy is even worse for maximum latency, where we observed that it increases rapidly; this is shown in Figure 2. This indicates that some jobs will eventually get stuck. In Figure 3 we analyzed the ratio and the difference between ScanALL to ScanOPT latencies, and both are increasing. This indicates that ScanALL is becoming worse over time, confirming the theory that ScanOPT stabilizes while ScanALL does not (c.f., Theorem 1 and Theorem 3, respectively, applied to the experiment setting of arrival rates $\lambda,\kappa$ and scanning vectors $\alpha^{*}$ and scan-all, respectively).

The figures measuring queue sizes over time show that the trend is in fact similar to the trend in latency with ScanALL performing considerably worse than ScanOPT while ScanNone grow even more rapidly over time. Figure  4 shows the average while queue sizes while Figure  5 shows the maximum queue sizes and Figure  6 shows the ratio and differences of ScanALL and ScanNONE increasing overtime.

7.2.2 Decentralized Approaches

In Figure 7, we compare the latency of the six decentralized algorithms using ScanOPT strategy, i.e., ScanOPT_JSQ, ScanOPT_JSW, ScanOPT_P2Q, ScanOPT_P2W, ScanOPT_UR and ScanOPT_RR. The best performing algorithm is the one based on JSW. This is followed by the two algorithms based on power of choices, and then the one based on JSW. The worst performing algorithms are the ones based on round robin and uniform random selection, which grow rapidly.

Figure 8, shows the trend for maximum latency. Where as expected, the algorithm based on JSW outperforms all the algorithms. A strange phenomenon we noticed is that of JSQ. We do not have a clear explanation of this phenomenon, although we suspect that this could be because choosing right configuration based on workload, as is done by RobustMaxWork, causes long windows of time without feeding the local queues using the shortest workload policy could make these windows (and thus latencies) even longer than using the shortest work paradigm.

In Figure 9 we compared the average queue sizes of the 6 decentralized algorithms using ScanOPT strategy i.e. ScanOPT_JSQ, ScanOPT_JSW, ScanOPT_P2Q, ScanOPT_P2W, ScanOPT_UR and ScanOPT_RR. The best performing algorithm is the one based on JSW it is not surprising because it considers the exact amount of work left in all the servers to decide where to route new jobs. This is followed by JSQ and then the two algorithms based on power of two choices. The worst performing algorithms are the ones based on round robin and uniform random selection which are not stabilizing. The performance is very similar for maximum queue size comparison shown in Figure 10.

Therefore the general trend is that the algorithms based on checking all the servers (JSQ and JSW) always outperform the ones based on power of two choices (P2W and P2Q), this phenomenon is not surprising because the algorithms based on all the servers consider the entire system state while the ones based on power of two choices are random. It should be note that the algorithms based on power of two choices are faster because decision can be made in constant time (which server to send an arriving job) while the ones based on all servers decision can only be made linear to the number of servers.

8 Conclusions, Extensions and Open Problems

8.1 Decentralized scheduling

We provided a rigorous mathematical analysis of a centralized scheduler RobustMaxWork with central queues, where jobs could be distributed to various machines at any time after their arrival. Similar analysis applies to the decentralized RobustMaxWork with Join-Shortest-Work (JSW) routing policy. Recall that this policy, upon arrival of type-$j$ job, sends it to machine that has minimum workload of type-$j$ jobs, $Z_{j}^{(m)}$, which is defined as in section 2 but now computed for each machine separately. Then each machine tries to maximize work done, $\max_{N\in S}\sum_{j=0}^{J}Z_{j}^{(m)}(t)N_{j}$. All the steps in the analysis of the main algorithm RobustMaxWork apply in this setting, resulting in almost identical analysis as in Section 4. An interesting open problem is to analyze mathematically the other five decentralized implementations of RobustMaxWork, and perhaps other similar decentralized algorithms.

8.2 Non-preemptiveness

In the centralized RobustMaxWork we assumed that in regular time intervals (at the beginning of each time step) all machines can be reconfigured — all jobs could be rescheduled and redistributed among the machines, where they will be further processed. In practice, interrupting execution of some jobs may be very costly. One may consider a model, where a job, once started on a machine, can not be paused or rescheduled for completion in a different time, nor processed on a different machine. The main idea in adapting RobustMaxWork to this model is to divide time into windows of length of $T$ time slots. $T$ should be large enough so that any job could be started at the start of the time window without breaking the above constraint. The algorithm will schedule jobs as previously with an additional constraint that only jobs that can be finished within a time window may be started. The stability analysis should remain the same, except that the margin for unstable arrival rates should be made a bit larger to accommodate potential losses of resources at the end of the time window. Thus the higher $T$ the better stability, though the latency may increase - studying this trade-off is an interesting open problem.

8.3 Algorithms without knowledge of arrival rates

Finding the optimal scanning frequencies requires the knowledge of arrival rates of jobs of each type, length and genuine/malicious status. In practice, however, these values are not provided in advance. We can estimate them given a large enough sample.

If arrival rates are not provided, we can start RobustMaxWork algorithm using scan-all strategy for a fixed but sufficiently long amount of time. During this time we learn the genuine/malicious status of jobs (due to the scan-all strategy) and therefore we will be able to estimate user-generated and malicious jobs arrival rates. We can then use scanning frequencies that are optimal for the estimated arrival rates. Note that using a different scanning strategy at the beginning for a fixed amount of time should not have impact on stability.

Furthermore, we can use scan-all strategy for a fixed amount of time repeatedly, with significantly longer pause after each time (during which we will be using the scanning probabilities computed based on the estimates), in order to enhance the quality of estimation of arrival rates, and thus using resources more and more efficiently. If pauses get long enough, this strategy should give better results than running scan-all strategy only once.

Another approach is to use scan-all strategy once and then run the algorithm with optimal scanning frequencies for calculated estimations, but utilize information given by scanning jobs according to optimal scanning frequencies. Since job arrivals are i.i.d. among time slots, scanning $x$ jobs randomly should give as good estimations as scanning first $x$ jobs. Therefore, even when using optimal scanning frequencies, we can improve our estimation of arrival rates each time a job was scanned. Designing and analyzing a stable algorithm for more scarce adversarial arrivals of malicious jobs is an interesting open problem.

8.4 Probability of successful scanning

We can also enhance the model by assuming that scanning can fail with some probability $p$, i.e., a malicious job may be scanned but still not discovered as a malicious one. In such model, if scanning failed on a malicious job, then this job is indistinguishable from a genuine user-generated one. Therefore with probability $p$ scanned malicious jobs will have to be processed fully. We should still be able to use RobustMaxWork algorithm, with a slight modification of the formula for job weights $Z_{j}$, which would have to include the failure probability as additional factor in some components. As the result, the optimal scanning frequencies may be slightly different but still in many cases better than scan-all or scan-none.