Speed Scaling with Tandem Servers

TL;DR

This paper studies speed scaling in tandem server networks, proposing online algorithms that are competitive with optimal solutions in both worst-case and stochastic scenarios, with power-efficient strategies.

Contribution

It introduces online algorithms for tandem server speed scaling that achieve constant competitiveness in worst-case and stochastic models, independent of workload and server count.

Findings

The proposed online algorithm is constant competitive in worst-case scenarios.

Random routing with static speed selection is constant competitive in stochastic settings.

Competitive ratio depends only on power functions, not workload or number of servers.

Abstract

Speed scaling for a tandem server setting is considered, where there is a series of servers, and each job has to be processed by each of the servers in sequence. Servers have a variable speed, their power consumption being a convex increasing function of the speed. We consider the worst case setting as well as the stochastic setting. In the worst case setting, the jobs are assumed to be of unit size with arbitrary (possibly adversarially determined) arrival instants. For this problem, we devise an online speed scaling algorithm that is constant competitive with respect to the optimal offline algorithm that has non-causal information. The proposed algorithm, at all times, uses the same speed on all active servers, such that the total power consumption equals the number of outstanding jobs. In the stochastic setting, we consider a more general tandem network, with a parallel bank of servers at each stage. In this setting, we show that random routing with a simple gated static speed selection is constant competitive. In both cases, the competitive ratio depends only on the power functions, and is independent of the workload and the number of servers.

Equations

$$C=\int n(t)dt+\int \sum _{k=1}^{K}P(s_k(t))dt.$$

$${\mathsf{c}}_A(\sigma )=\frac{C_A(\sigma )}{C_{\mathsf{OPT}}(\sigma )},$$

$$n(t)+\sum _{k=1}^{K}P(s_k(t))+\frac{d\Phi (t)}{dt}\le c(n_o(t)+\sum _{k=1}^{K}P(s_k^o(t))),$$

$$\left(\int n(t)+\sum_{k=1}^{K}P(s_{k}(t))\right)\leq\int c\biggl{(}n_{o}(t)+\sum_{k=1}^{K}P(s^{o}_{k}(t))\biggr{)},$$

$$C_A\le$$

$$d^j(t,q)=\sum _{k=2}^{j-1}{n}^k(t)+n^j(t,q),$$

$$f_a\left(\frac{i}{a}\right)-f_a\left(\frac{i-1}{a}\right)=\Delta \left(\frac{i}{a}\right):=P^{\prime }\left(P^{-1}\left(\frac{i}{a}\right)\right)$$

$${\Phi }_1(t)=c{\int }_0^1{f}_1\left(d^1(t,q)\right)dq,$$

$${\Phi }_j(t)={\Phi }_1(t)+{\Phi }_j(t)$$

$${\Phi }_j(t)=c_j{\int }_0^1{f}_{A(t)+1}\left(\frac{d^j(t,q)}{A(t)+1}\right)dq.$$

$$\Phi (t)=\sum _{j=1}^K{\Phi }_j(t),$$

$$s_1(t)=\{\begin{array}{ll}{P}^{-1}\left(\frac{n^1(t)+A(t)+1}{A(t)+1}\right),& \text{if}{n}^1(t)>0,\\ 0,& \text{otherwise.}\end{array}$$

$$s_i(t)=\{\begin{array}{ll}{P}^{-1}\left(\frac{n^1(t)+A(t)+1}{A(t)+1}\right),& \text{if}{n}^1(t)>0,\\ P^{-1}(2),& \text{otherwise.}\end{array}$$

$${\Phi }_j(t)=c_j{\int }_0^1{f}_{A(t)+1}\left(\frac{n^1(t)+{\sum }_{k=2}^{j-1}(n^k(t))+n^j(t,q)}{A(t)+1}\right)dq,$$

$$d^k(t,q)$$

$$d{\Phi }_1/dt\le \{\begin{array}{ll}cP(s_o)-c\frac{(n_1-n_o)}{K}& \text{if}{n}_o<n^1,\\ 0& \text{if}{n}_o>n^1.\end{array}$$

$$d{\Phi }_i$$

$$n^1+A+\sum _{k=1}^{K}P(s_i(k))+d\Phi /dt$$

$$A+\sum _{k=1}^{K}P(s_i(k))+d\Phi /dt$$

$$n^1+A+\sum _{k=1}^{K}P(s_i(k))+d\Phi /dt$$

$$\le c{n}_o+(2-c)n^1+A(2-c/2)+1+c\sum _{k=1}^{K}P(s_o),$$

$$\le 6n_o+6\sum _{k=1}^{K}P(s_o),$$

$$\int n^1+A+\sum _{k=1}^{K}P(s_i(k))+d\Phi /dt\le 6\left(\int \left(n_o+\sum _{k=1}^{K}P(s_o)\right)dt\right)+12\Delta (1)(nK),$$

$$C_A\le 6C_{\mathsf{OPT}-E}+12\Delta (1)nK.$$

$$C_A$$

$$C=\mathbb{E}\left[T\right]+\mathbb{E}\left[E\right],$$

$$C=\sum _{i=1}^K\left(\mathbb{E}\left[T_i\right]+\mathbb{E}\left[E_i\right]\right).$$

$${\Phi }_j$$

$$\le c_j{\int }_0^1\left\{f_{A(t)+1}\left(\frac{d^j(t,q)+1}{A(t)+1}\right)-f_{A(t)+1}\left(\frac{d^j(t,q)}{A(t)+1}\right)\right\}dq,$$

$$=c_j\Delta \left(\frac{d^j(t,q)+1}{A(t)+1}\right),$$

$$\le c_j\Delta (1),$$

$${\Phi }_j$$

Full text

1. Introduction

Starting with the classical work [26], the speed scaling problem has been widely considered in literature, where there is a single/parallel bank server with tuneable speed, and the canonical problem is to find the optimal service speed/rate for servers that minimizes a linear combination of the flow time (total delay) and total energy [1], called flow time plus energy, where flow time is defined as the sum of the response times (departure minus the arrival time) across all jobs.

Many interconnected practical systems such as assembly lines, flow shops and job shops in manufacturing, traffic flow in a network of highways, multihop telecommunications networks, and client-server computer systems, however, are better modelled as network of queues/servers. Another important example is service systems with server specific precedence constraints, where jobs have to be processed in a particular order of servers. In such systems, for each job, service is defined to be complete once it has been processed by a subset of servers, together with a permissible order on service from different servers.

The simplest such network is a $K$-server tandem setting, where there are $K$ servers in series, and each object/job has to be processed by all the $K$ servers in a serial order. With $K$-tandem servers, we consider the speed scaling problem of minimizing flow time plus energy, when the speed/service rate of each server is tuneable and there is an associated energy cost attached to the chosen speed. The control variables here include scheduling, i.e., which job to run on each server, and speed scaling, i.e., which speed to operate each server at. In the worst case setting, the arrival sequence is arbitrary, and possibly adverserially determined. In this case, the performance metric is the competitive ratio, that is defined as the maximum of the ratio of the cost of the online algorithm and the optimal offline algorithm $\mathsf{OPT}$ that is allowed to know the entire input sequence ahead of time, over all possible inputs. In the stochastic setting, job arrivals occur according to a stochastic process. Here, the cost of an algorithm is the sum of the steady state averages of response time and energy consumption per job. The competitive ratio of an algorithm is in turn the ratio of its cost to that of the optimal algorithm (that admits the above steady state averages). In both settings, the goal is to design algorithms that have a small competitive ratio.

1.1. Related Work

1.1.1. Arbitrary Input Case

With arbitrary input, where job arrival times and sizes are arbitrary (even chosen by an adversary), for speed scaling with a single server or bank of parallel servers, two classes of problems have been studied: i) unweighted and ii) weighted, where in i) the delay that each job experiences is given equal weight in the flow time computation, while in ii) it is scaled by a weight that can be arbitrary.

The weighted setting is fundamentally different that the unweighted one, where it is known that constant-competitive online algorithms are not possible [6], even for a single server, while constant competitive algorithms are known for the unweighted case, even for arbitrary energy functions, e.g., the best known $2$-competitive algorithm [3]. For more prior references on speed scaling, we refer the reader to [3, 8].

In addition to a single server, speed scaling problem has also been considered extensively for a parallel bank of servers, where there is a single queue and multiple servers. With multiple servers, on an arrival of a new job, the decision is to choose which jobs to run on the multiple servers, by preempting other jobs if required, and what speed [19, 14, 16, 21, 15]. The homogenous server case was studied in [21, 14], i.e., power usage function is identical for all servers, while the heterogenous case was addressed in [16, 15, 11], where power usage function is allowed to be different for different servers.

1.1.2. Stochastic Input Case

Under stochastic input, research on two tandem servers with variable speed was initiated in the classical work of [23] [17], that established that the optimal service rate of the first queue is monotonic in the joint queue state, and is of the bang-bang type. These results have also been extended for any number of tandem servers when each server has exponential service distribution [24]. These type of problems belong to the class of control of Markov decision processes for which general results have been also derived [13]. Typically, in early works, the objective function did not include an energy cost for increasing the speed of the service rate. To reflect the energy cost, [25] considered the same problem as in [23] in the presence of an average power constraint. Analytical results in this area have been limited to structural results, such as the monotonicity results, and that too for special input/service distributions, and no explicit optimal service rates are known. In the stochastic setting, with multiple parallel servers, the flow time plus energy problem with multiple servers is studied under a fluid model [5, 22] or modelled as a Markov decision process [12], and near optimal policies have been derived.

1.2. Our work

We consider the speed scaling problem in the tandem network setting, where there are multiple servers ($K$) in series. Each external job arrives at server $1$, and is defined to be complete once it has been processed by each of the $K$ servers in series. Each server has an identical power (energy) consumption function $P(.)$, i.e., if the server speed is $s$, then power consumed is $P(s)$.

1.2.1. Arbitrary Input

We consider the arbitrary input setting, where jobs can arrive at arbitrary time on server $1$, arrival times possibly chosen by an adversary. However, we assume that each job has the same size/or requirement on any of the servers. Even for a single server setting, initial progress was made for unit sized jobs [1, 20, 7, 10, 2], which was later generalized for arbitrary job size. In the sequel, it is evident that the considered problem is challenging even with unit sized jobs. The motivation to consider the arbitrary input setting is two-fold : i) that it is the most general, ii) that even if one assumes that the external arrivals to server $1$ have a nice distribution, with speed scaling by each of the server, the internal arrivals (arrivals at server $k$ corresponding to departures from server $k-1$) need not continue to have the same nice distribution. Under the arbitrary input setting, we consider the unweighted flow time + energy as the objective function, and the problem is to find an online algorithm with minimum competitiive ratio.

The proposed algorithm ensures that there is at most one outstanding job with all servers other than server $1$. Let $n^{1}(t)$ be the number of outstanding jobs with server $1$, and let the total number of servers with outstanding jobs (called active) be $A(t)$ excluding the first server. Then the algorithm runs each active server (including server $1$) at the same speed of $P^{-1}\left(\frac{n^{1}(t)+A(t)+1}{A(t)+1}\right)$. Thus, the total power consumed across all servers is equal to the number of total outstanding jobs plus $1$, that could be spread across servers. The main result of this paper is as follows.

Theorem 1.1**.**

With unit sized jobs, and identical power consumption function $P$ for all servers, the competitive ratio of the proposed algorithm is at most $\left(6+\left(12/P(s^{\star})\right)\Delta(1)\right)$ where $\Delta(1)=P^{\prime}(P^{-1}(1))$ and $1+P(s^{\star})=s^{\star}P^{\prime}(s^{\star})$. For $P(s)=s^{\alpha}$, $\Delta(1)=\alpha$ and for $\alpha=2$, $P(s^{\star})=2$, making the competitive ratio at most $18$.

Even though there has been large number of papers on online speed scaling algorithms with a single server or with multiple parallel servers, as far as we know, there is no work on competitive algorithms for a tandem server case for the objective of flow time plus energy. We would like to point that there is work on only energy efficient routing for networks [4, 9] without any delay consideration.

With a tandem network, the main technical difficulty in obtaining results for flow time plus energy with the arbitrary input case is that the external arrivals happen at the same time for any algorithm and the optimal offline algorithm $\mathsf{OPT}$ into server $1$, but because of dynamic speed scaling, the internal arrivals at intermediate servers (departures from previous server) are not synchronized for any algorithm and the $\mathsf{OPT}$ (that has non-causal information about future job arrivals). Thus, a sample path result that is needed in the arbitrary input case is hard to obtain.

We overcome this difficulty by proposing a potential function that has positive jumps (corresponding to movement of jobs in consecutive servers) in contrast to typical approach of using potential function that has no positive jumps. Consequently, to derive constant competitive ratio results, we upper bound the sum of the positive jumps and relate that to the cost of the $\mathsf{OPT}$. Moreover, the potential function is only a function of the number of jobs with the $\mathsf{OPT}$ in the first server and not in any subsequent servers, since controlling and synchronizing the jobs of the algorithm and the $\mathsf{OPT}$ in servers other than the first is challenging. We show in Remark 3.1, that a simple/natural extension of the the popular speed scaling algorithm [8] for a single server, does not yield any useful bound on the competitive ratio with tandem servers. Our result is similar in spirit to the results of [16, 11] for parallel servers, where the competitive ratio also depends on $P(.)$. Compared to the prior work on speed scaling with single(parallel) server(s) [8, 16, 11], we make a non-trivial extension (even though our results require unit sized jobs) and provide constant competitive ratio results for tandem servers, that has escaped analytical tractability for long.

1.2.2. Stochastic Input

In the stochastic setting, we consider a more general tandem network, with a parallel bank of servers at each stage. The external arrivals to stage $1$ are assumed to follow a Poisson distribution. We consider a simple ’gated’ static speed algorithm and random routing among different servers in each stage that critically ensures that the job arrivals to subsequent stages are also Poisson [18]. We show that the random routing and gated static speed policy has a constant competitive ratio that only depends on the power functions, and is independent of the workload and the number of servers. To contrast our work with prior work on stochastic control of tandem servers [23, 17], the novelty of our work is that we are able to give concrete (constant factor) competitive ratio guarantees, while in prior work only structural results were known that too in the stochastic input setting.

2. System Model

Let the input consist of $n$ jobs, where job $j$ arrives (released at) at time $a_{j}$ and has work/size $w_{j}^{k}$, to be completed on server $k$. There are $K$ homogenous servers in series/tandem, each with the same power function $P(s)$, where $P(s)$ denotes the power consumed by any server while running at speed $s$. Typically, $P(s)=s^{\alpha}$ with $2\leq\alpha\leq 3$. Each job has to be processed by each of the $K$ servers, in sequence, i.e. server $k$ can process a job only after it has been completely processed by server $k-1$ and departed from it. Following most of the prior work in this area, we assume that each server has a large enough buffer and no jobs are dropped ever.

The speed $s$ is the rate at which work is executed by any of the server, and $w$ amount of work is completed in time $w/s$ by any server if run at speed $s$ throughout time $w/s$. A job $j$ is defined to be complete at time $f_{j}$ on server $k$ if $w_{j}^{k}$ amount of work has been completed for it on server $k$. The flow time $F_{j}$ for job $j$ is defined as $F_{j}=f_{j}-a_{j}$ ($f_{j}$ is the completion time of job $j$ on the last ($K^{th}$) server minus the arrival time) and the overall flow time is $F=\sum_{j}F_{j}$. From here on we refer to $F$ as just the flow time. Note that $F=\int n(t)dt$, where $n(t)$ is the number of unfinished jobs (spread across possibly different servers) at time $t$. We denote the corresponding variables for the the optimal offline algorithm $\mathsf{OPT}$ by a subscript or superscript $o$.

Let server $k$ run at speed $s_{k}(t)$ at time $t$. Then the energy cost for server $k$ is defined as $P(s_{k}(t))$, where $P(.)$ is strictly convex, non-decreasing, differentiable function at $s\in[0,\infty)$. Natural example of $P(x)=a+bx^{\alpha},\alpha,a,b\geq 1$ clearly satisfies all these conditions. Following [8], these special conditions on $P$ can be relaxed completely, without affecting the results, and more importantly work even when maximum speed is bounded $s\in[0,\mathsf{B}]$ (see Remark 6.3). Total energy cost is $\sum_{k=1}^{K}P(s_{k}(t))$ summed over the flow time.

Choosing larger speeds reduces the flow time, however, increases the energy cost, and the natural objective function that has been considered extensively in the literature is a linear combination of flow time and energy cost, which we define as

[TABLE]

Note that there is no explicit need for considering the weighted combination of the two costs in (1) since a scalar multiple can be absorbed in the power function $P(.)$ itself.

3. Arbitrary Input

Any online algorithm only has causal information, i.e., it becomes aware of job $j$ only at time $a_{j}$. Using only this causal information, any online algorithm has to decide at what speed each server should be run at at each time. Let the cost (1) of an online algorithm $A$ be $C_{A}$, and the cost for the $\mathsf{OPT}$ that knows the job arrival sequence $\sigma$ (both $a_{j}$ and $w_{j}^{k}$) in advance be $C_{\mathsf{OPT}}$. Then the competitive ratio of the online algorithm $A$ for $\sigma$ is defined as

[TABLE]

and the objective function considered in this paper is to find an online algorithm that minimizes the worst case competitive ratio ${\mathsf{c}}^{\star}=\min_{A}\max_{\sigma}{\mathsf{c}}_{A}(\sigma)$.

A typical approach in speed scaling literature to upper bound (of $c$) the competitive ratio is via the construction of a potential function $\Phi(t)$ and show that for any input sequence $\sigma,$

[TABLE]

almost everywhere and that $\Phi(t)$ satisfies the following boundary conditions,

1. (1)

Before any job arrives and after all jobs are finished, $\Phi(t)=0$, and 2. (2)

$\Phi(t)$ does not have a positive jump discontinuity at any point of non-differentiability.

Then, integrating (3) with respect to $t$, we get that

[TABLE]

which is equivalent to showing that $C_{A}(\sigma)\leq c\ C_{\mathsf{OPT}}(\sigma)$ for any input $\sigma$ as required.

Since any online algorithm is only allowed to make causal decisions, thus at any time $t$, the speed chosen by an online algorithm $A$ for any server and the $\mathsf{OPT}$ can be different. Because of this, the main challenge when there are tandem servers, is that the internal arrivals at server $k+1$ that corresponds to departures from server $k$ ($k<K$ other than the last) can happen at different times for the algorithm and the $\mathsf{OPT}$. Thus constructing a potential function and ensuring that the boundary conditions are satisfied presents a unique challenge. With a single (or parallel bank) server such a problem does not arise since there, arrivals only happen externally at the same time for both the algorithm and the $\mathsf{OPT}$. Thus, instead of finding a potential function that does not have a positive jump discontinuity, we propose a potential function for which we can control how large the positive jump discontinuity and compare it with cost of the $\mathsf{OPT}$. Let the new boundary conditions be,

1. (1)

Before any job arrives and after all jobs are finished, $\Phi(t)=0$, and 2. (2)

Let $\Phi(t)$ increase by amount $D_{j}$ at the $j^{th}$ discontinuous point. Let $\sum_{j}D_{j}\leq\mathsf{D}C_{\mathsf{OPT}}$.

Then, integrating (3) with respect to $t$, we get that

[TABLE]

which is equivalent to showing that $C_{A}(\sigma)\leq(c+D)\ C_{\mathsf{OPT}}(\sigma)$ for any input $\sigma$ as required.

The main novel contribution of this paper is the construction of a potential function for tandem server settings with positive jumps, where we can upper bound $\mathsf{D}$, and importantly which is only a function of the number of jobs with the $\mathsf{OPT}$ on the first server (which arrive together for the algorithm as well) and not on subsequent servers, since controlling them is far too challenging.

Job sizes: For a single server setting, constant competitive algorithms have been derived independent of the job sizes [8]. Considering arbitrary job sizes in a multiple tandem server setting is more complicated (technical difficulty is described in Remark 3.7) and we consider homogenous job size setting, where all job sizes are identical across all jobs and all servers $w_{j}^{k}=w,\ \forall\ j,k$. Without loss of generality, we infact let $w=1$.

We here discuss briefly why it is non-trivial to extend the results for single server setting to tandem server setting.

Remark 3.1**.**

Let $w=1$. Consider a $c$-competitive algorithm $A_{c}$ for a single server with equal job size, e.g. $c=3$ [8] that chooses speed $s=P^{-1}(n+1)$, where $n$ is the number of outstanding jobs. There are two ways to use this in the tandem server setting. Let $n^{i}$ be the number of jobs on server $i$. Either we can replicate the speed of jobs as seen on server $1$ ($s_{1}=P^{-1}(n^{1}+1)$) on server $2$, or use $s_{i}=P^{-1}(n^{i}+1),i=1,2$ autonomously on both the servers. We argue next that both these choices are not very useful.

Speed Replication:* Let job $j$ arrive at time $a_{j}$ and depart at $f_{j}$, and during this time the speed chosen by server $1$ to serve job $j$ be $s_{j},t\in[f_{j},a_{j}]$. Replicating the speed profile $s_{j}$ on the second server as well does not result in $2c$-competitive algorithm for the two-server problem. What can happen is that consider a time $t$ where a job $j$ starts its service at server $2$ and let that job $j$ was alone in server $1$ throughout the time it spent in server $1$, i.e. its speed profile $s_{j}=P^{-1}(2),t\in[f_{j},a_{j}]$ [8]. Let the next job $j+1$ arrive at $t=f_{j}$ into server $1$. The speed of job $j+1$ in server $1$ is $P^{-1}(2)$, and because of replication of $s_{j}$ on server $2$ for job $j$, job $j$ is also being processed at speed $P^{-1}(2)$. Let at $t^{+}$, $n>>1$ new jobs arrive in server $1$, because of which the speed of job $j+1$ is increased to $P^{-1}(n+2)$. Thus, with the $2$-server setting, job $j+1$ will be processed fast and will have to wait behind job $j$ in server $2$ since job $j$’s speed is fixed at $P^{-1}(2)$. Such a problem is avoided in a single server system since at time $t^{+}$ job $j$ has departed the system. Thus, with the two-server system, the cost for job $j+1$ could be more than two times compared to a single server system.*

*Autonomous: *** Consider an input, where $\ell>>1$ jobs of unit size arrive at time [math] into server $1$. Then choosing $s_{i}=P^{-1}(n^{i}+1),i=1,2$, server $2$ runs slower compared to server $1$ until $\ell/2$ jobs have been processed by server $1$ and are available at server $2$. Thus, jobs start accumulating in server $2$’s queue, and consequently, each of $\ell$ jobs have to wait behind jobs in server $2$ for sufficient time before they are processed by server $2$, entailing a large flow time + energy cost. This argument on its own does not mean that the competitive ratio of this algorithm is poor, since the inherent cost could be large even with the $\mathsf{OPT}$ with this input. However, for this input, instead a simple algorithm ($\mathsf{OPT}$ can only do better) that chooses $s_{i}=P^{-1}(n^{1}+1)$ for both $i=1,2$ avoids any waiting for any job on server $2$ and can be shown to have at most twice the flow time + energy cost of the server $1$. Thus autonomous speed choice for two servers is also not expected to provide low (or constant) competitive ratio.

3.1. Speed Scaling Algorithm

We begin this section, by first deriving a lower bound on the cost of the $\mathsf{OPT}$.

Lemma 3.2**.**

$C_{\mathsf{OPT}}\geq C_{\mathsf{OPT}-E}$, where $C_{\mathsf{OPT}-E}=\int\left(n^{1}_{o}(t)+\sum_{k=1}^{K}P(s^{1}(t))\right)dt$.

Proof 3.3**.**

We enhance the $\mathsf{OPT}$ as follows to derive a lower bound on its cost. Instead of requiring that $\mathsf{OPT}$ processes jobs in series, each incoming job is copied on all servers and a job is defined to be complete, when it is completed by all servers. Thus, allowing $\mathsf{OPT}$ to run jobs in parallel. Essentially this will let $\mathsf{OPT}$ run jobs at same speed in each of the servers, and have the same number of outstanding jobs on each server. Thus, for the enhanced $\mathsf{OPT}$, the total cost (flow time + energy) is equal to $C_{\mathsf{OPT}-E}=\int\left(n^{1}_{o}(t)+\sum_{k=1}^{K}P(s^{1}(t))\right)dt$, where $n^{1}_{o}(t)$ is the number of outstanding jobs on server $1$. Thus, we have $C_{\mathsf{OPT}}\geq C_{\mathsf{OPT}-E}$.

Next, we will compare the performance of the proposed algorithm and the enhanced $\mathsf{OPT}$. Let $n^{k}(t)$ and $n_{o}^{k}(t)$ the number of outstanding jobs on server $k$ with the algorithm and the enhanced $\mathsf{OPT}$ (which for succinctness call $\mathsf{OPT}$ whenever there is no ambiguity), respectively at time $t$. For the enhanced $\mathsf{OPT}$ we only need to consider the number of jobs on server $1$. At time $t$, let $n_{o}(t,q)$ be the number of unfinished jobs with $\mathsf{OPT}$ on the first server with remaining size at least $q$, while $n^{i}(t,q)$ be the number of unfinished jobs with the algorithm on server $i$ with remaining size at least $q$. Thus, $n^{i}(t)=n^{i}(t,0)$ and $n_{o}(t)=n_{o}(t,0)$.

For server $1$, let $d^{1}(t,q)=\max\left\{0,\frac{n^{1}(t,q)-n_{o}(t,q)}{K}\right\}$, while for server $j\geq 2$,

[TABLE]

where $\sum_{k=2}^{j-1}n^{k}(t)$ is the total number of outstanding jobs from server $2$ till server $j-1$. Notably in defining $d^{j}(t,q)$ there is no contribution from the $\mathsf{OPT}$ unlike in $d^{1}(t,q)$. This is key, since there is no way to control the number of jobs that the $\mathsf{OPT}$ has in server $j\geq 2$ and their transitions between servers $j$ to $j+1$.

For the algorithm, a server $i$ is defined to be active if it has an unfinished job, i.e., $n^{i}(t)>0.$ The indicator function $A_{i}(t)=1$ for $i\geq 2$ if server $i$ is active under the algorithm at time $t$ and $A_{i}(t)=0$ otherwise. Then $A(t)=\sum_{i=2}^{K}A_{i}(t)$ is the number of active servers with the algorithm at time $t$, other than server $1$.

For $i\in\mathbb{N},$ let

[TABLE]

and $f_{a}(0)=0$. For server $1$, let

[TABLE]

while for server $j\geq 2$,

[TABLE]

where

[TABLE]

Consider the potential function

[TABLE]

**Algorithm: ** The speed scaling algorithm that we propose, chooses the following speeds. For server $1$,

[TABLE]

For active servers, i.e., servers $i\geq 2$ with $A_{i}=1$

[TABLE]

The non-active servers have speed [math].

With this speed scaling choice, all active servers work at the same speed at each time, and since we are assuming that each job has the same size on all servers, this implies that jobs only wait in server $1$ if at all, and are always in process at active servers $i>1$. Moreover, other than server $1$, all servers have at most $1$ outstanding job at any time. The speed choice ensures that the total power used is $n^{1}+A+1$ (or $2A$ if $n^{1}=0$) one more than the total number of outstanding jobs in the system.

Comments about the potential function: The basic building blocks $\Phi_{1}$ and $\Phi_{j}$* of our proposed potential function are inspired by the potential function first constructed in [8], however, the non-trivial aspect is the choice of including $A(t)$ to define the $f$ function. Since $A(t)$ changes dynamically, the overall construction and analysis is far more challenging.*

The proposed potential function $\Phi$ is rather delicate and is really the core idea for solving the problem. We discuss its important properties and reasons why a more natural choice does not work as discussed in Remark 3.4. To begin with, note that the denominator in $d^{1}(t,q)$ is fixed to be $K$ and not $A(t)$ which can change dynamically. This is important since $n^{1}(t)$ can be arbitrarily large, and a decrease in $A(t)$ can have an arbitrarily large increase in $\Phi_{1}(t)$. However for $d^{j}(t,q)$ which is function of $A(t)$, even when $A(t)$ decreases, the increase in $\Phi_{j}(t)$ can be bounded since $n^{j}(t)\leq 1$ (choice made by the proposed algorithm) and $\sum_{j\geq 2}n^{j}(t)\leq A(t)$. The choice of potential function is also peculiar since $\Phi_{1}(t)$ is spread over all the $K$ servers with a normalization factor of $K$ (as defined in $d^{1}(t,q)$). This is needed since the algorithm prescribes an identical speed of $P^{-1}\left(\frac{n^{1}(t)+A(t)+1}{A(t)+1}\right)$ for all the servers, and to get sufficient negative drift from the $d\Phi_{1}(t)/dt$ term, it is necessary that $P\left(P^{-1}\left(\frac{n^{1}(t)+A(t)+1}{A(t)+1}\right)\right)\geq n^{1}/K$, which is true since $A(t)\leq K-1$. If instead we just keep one term for $\Phi_{1}(t)$ in $\Phi(t)$ without the normalization by $K$ in $d^{1}(t,q)$, the speed of each server has to be at least $P^{-1}(n^{1}(t))$ to get sufficient negative drift from the $d\Phi_{1}(t)/dt$ term, however, that makes the total power used $\sum_{j=1}^{n}P^{-1}(s_{j})=Kn^{1}(t)$, which is order wise too large.

Remark 3.4**.**

The considered potential function (6) for server $j$ is not a natural choice. Instead it should really be

[TABLE]

by combining the arguments of $\Phi_{1}$ and $\Phi_{j}$ into a single $f$ function. This choice avoids the increase in $\Phi_{j}$ when a job moves from server $k$ to $k+1$ unlike (6), since in this case $n^{k}(t^{+})=n^{k}(t)-1$, while $n^{k+1}(t^{+},q)=n^{k+1}(t,q)+1$ cancelling each other off. This, however, makes controlling the increase in $\Phi_{j}(t)$ when $A(t)$ decreases, since $n^{1}(t)$ can be arbitrarily large. The current choice (6) avoid this bottleneck by isolating server $1$ from all the other subsequent servers by keeping the terms of server $1$ and subsequent servers (6) separate, however, at a cost of incurring positive jumps whenever jobs move from server $k$ to $k+1$ which can be bounded.

Remark 3.5**.**

To eliminate the need for considering different epochs at which the job transition happens between server $k$ and server $k+1$ with the algorithm and the $\mathsf{OPT}$ which can result in increase in the potential function, one can consider a following equivalent model. Let on (external) arrival of a new job $j$ to server $1$ at time $t$, $K$ jobs are created with sizes $w$, and the $k^{th}$ copy with size $w$ is sent to the $k^{th}$ server at time $t$. To model the tandem server constraint, a precedence constraint can be enforced such that any copy of any job cannot start its processing at server $k$ unless it has been processed (served and departed) at the server $k-1$. The precedence constraint, however, brings in a new feature unlike the single server case, that the servers can idle even when they have outstanding jobs, if those jobs have not been processed by preceding servers, which needs to be handled carefully.

Following [8], a natural choice for the potential function with this alternate model is $\Phi_{1}(t)=c_{1}\int_{0}^{\infty}f\left(d^{1}(t,q)\right)dq$, and $\Phi_{k}(t)=c_{k}\int_{0}^{\infty}f\left(d^{k}(t,q)\right)dq$, and consider the potential function $\Phi(t)=\Phi_{1}(t)+\sum_{k=2}^{K}\Phi_{k}(t)$, where $d^{1}(t,q)=\max\left\{0,n^{1}(t,q)-n^{1}_{o}(t,q)\right\}$, and

[TABLE]

and $f(0)=0$, and $\forall\ i\geq 1$, $f(i)-f(i-1)=\Delta(i):=P^{\prime}(P^{-1}(i))$ (this means $P^{\prime}(x)$ where $x=P^{-1}(i)$). To get the correct negative drift with this potential function, however, requires $c_{k}>c_{k+1}$ because of the 'back flow' (terms of type $n^{k}(t,q)-n^{k-1}(t,q)$ in $\Phi_{k}$ which increase the potential function $\Phi_{k}$ when the algorithm is working on server $k-1$) making $c_{1}\geq K$, and since the competitive ratio at least $c_{i}$ for all $i$, the resulting competitive ratio turns out to be $K$.

From here on we work towards proving Theorem 1.1. The first step in that direction is to bound the increase in the potential function $\Phi(t)$ at discontinuous points, which is done as follows.

Lemma 3.6**.**

Taking $c_{j}=c$ for all $j\geq 2$, the total increase in $\Phi(t)$ at points of discontinuity is at most $2cnK\Delta(1)$.

Proof can be found in Appendix 5.

Remark 3.7**.**

The restriction of equal job sizes is essentially needed to prove Lemma 3.6. Since all server speeds are identical, if job sizes are different, jobs will accumulate in servers other than $1$, making it hard to control the increase in $\Phi(t)$ when $A(t)$ decreases.

Definition 3.8**.**

Let $r_{i}=\ell$, if $i$ is the $\ell^{th},\ell\in[1:A(t)]$ active server (in increasing order of server index) among the $A(t)$ active servers.

The proof of Theorem 1.1 is based on the following bounds on the potential function drift.

Lemma 3.9**.**

Consider any instant $t$ when no arrival/departure (including internal transfers) occurs under the algorithm or $\mathsf{OPT}.$ For server $1$,

[TABLE]

If $n_{o}(t)=n^{1}(t)$, then either of the above two cases arise. Moreover, for any active server $i\geq 2$ at time $t$,

[TABLE]

*where $A_{i}(t)=1$ when server $i\geq 2$ is active and zero otherwise. *

Proof can be found in Appendix 6.

Next, we consider the cost of the algorithm at any time $t$, and suppress $(t)$ for simplicity. When $n_{o}>n^{1}$ and $n^{1}\neq 0$, then $d\Phi_{1}/dt=0$ (Lemma 3.9), and since $\sum_{i=2}^{K}\frac{r_{i}}{A+1}\geq A/2$, the `running' cost (3) from Lemma 3.9 with $c_{j}=c,\ \forall\ j\geq 2$ is

[TABLE]

choosing $c=6$. If $n^{1}=0$, where each active server other than server $1$ has speed $P^{-1}(2)$, then the running cost

[TABLE]

choosing $c=6$. When $n_{o}<n^{1}$, then $d\Phi_{1}/dt\leq P(s_{o})-c\frac{(n_{1}-n_{o})}{K}$ using Lemma 3.9. Moreover, from Lemma 3.9 with $c_{j}=c,\ \forall\ j\geq 2$, where $\sum_{i=2}^{K}\frac{r_{i}}{A+1}\geq A/2$, the running cost,

[TABLE]

choosing $c=6$. Thus, in both cases, accounting for the discontinuities from Lemma 3.6 with $c_{j}=c=6$ for all $j$ since the first boundary condition is trivially met,

[TABLE]

which implies that

[TABLE]

Now we complete the Proof of Theorem 1.1.

Proof 3.10**.**

From (10)

[TABLE]

Recall that $C_{\mathsf{OPT}-E}\leq C_{\mathsf{OPT}}$. For any job with size $w$, the minimum cost incurred by $\mathsf{OPT}$ on processing it on any one server is $\min_{s}\frac{w}{s}+\frac{wP(s)}{s}$, where $s$ is the speed. Thus, the optimal $s^{\star}$ satisfies $1+P(s^{\star})=s^{\star}P^{\prime}(s^{\star})$, and the optimal cost is $wP^{\prime}(s^{\star})$. With $n$ jobs arriving each with size $1$ which have to be processed by each of the $K$ servers, a simple lower bound on the cost of $\mathsf{OPT}^{1}$ is $nKP^{\prime}(s^{\star})$. This implies from (11) that

[TABLE]

For $P(s)=s^{2}$, $s^{\star}=1$ and the minimum cost is $P^{\prime}(s^{\star})=2$, and $\Delta(1)=2$, thus $C_{A}\leq 18C_{\mathsf{OPT}}$.

4. Stochastic setting

In this section, we move from the worst case setting to the stochastic setting, where the workload is specified by stochastic processes and we evaluate algorithms based on their performance in steady state. We find that the stochastic setting is `easier' than the worst case setting; specifically, we show that a naive routing strategy coupled with a simple ON/OFF (gated static) speed selection is constant competitive. Crucially, the competitive ratio depends only on the power functions, and not on the statistical parameters of the workload or the topology of the queueing system. Moreover, the tandem system we consider in this section is more general that the one considered before—each job needs to be served in $K$ tandem layers/phases, where each layer $i$ is composed of $m_{i}$ parallel servers.

Formally, our system model is as follows. The service system is composed of $K$ tandem layers of servers, with $m_{i}$ parallel and identical servers in layer $i.$ Jobs arrive to layer $1$ according to a Poisson process with rate $\lambda.$ The jobs have to be processed sequentially in the $K$ layers (by any server in each layer) before exiting the system. Moreover, we assume that each server is equipped with an (infinite capacity) queue, so that once a job completes service in layer $i,$ $1\leq i\leq K-1,$ it can be immediately dispatched to any server in layer $i+1.$ The service requirement in layer $i$ is exponentially distributed with mean $1/\mu_{i}.$ Job scheduling on each server is assumed to be blind to the service requirements of waiting jobs. The power function for all servers in layer $i$ is $P_{i}(s)=c_{i}s^{\alpha_{i}},$ where $c_{i}>0,$ $\alpha_{i}>1.$

The performance metric is given by

[TABLE]

where $T$ and $E$ denote, respectively, the response time and energy consumption associated with a job in steady state. We note that the performance metric can be also be expressed as the sum of the costs incurred in each layer:

[TABLE]

Here, $T_{i}$ denotes the steady response time in layer $i,$ and $E_{i}$ denotes the steady state energy consumption (per job) in layer $i.$111We implicitly restrict attention to the class of policies that admit these stationary averages.

The proposed algorithm ($A$) is the following. When a job arrives into layer $i,$ we dispatch it to a random server in layer $i,$ chosen uniformly at random. The speed of each server in layer $i$ is set in a gated static fashion as $s_{i}^{A}=1+\frac{\rho_{i}}{m_{i}}$ when active (and zero when idle), where $\rho_{i}:=\frac{\lambda}{\mu_{i}}$ is the offered load to layer $i.$ Note that the speed selection requires knowledge (via learning if not available) of the offered load into each layer (unlike the dynamic speed scaling algorithm in (8). This boils down to learning the arrival rate and the mean service requirement, which is feasbile in the stochastic workload setting considered here. Under the proposed random routing and gated static speed selection, the system operates as a (feedforward) Jackson network, with each server operating as an M/M/1 queue **[18]**. Thus, the arrival process for each layer is also Poisson.

Our main result is the following. Let $[K]:=\{1,2,\cdots,K\}.$

Theorem 4.1**.**

The algorithm $A$ is constant competitive, with a competitive ratio that depends on only the power functions, i.e., on \bigl{(}(c_{i},\alpha_{i}):\ i\in[K]\bigr{)}. Specifically, the competitive ratio does not depend the workload parameters $\lambda,$ $(\mu_{i},i\in[K])$, the number of layers $K,$ or on the number of servers in the different layers $(m_{i},i\in[k]).$

The proof of Theorem 4.1 can be found in Appendix 7.

5. Proof of Lemma 3.6

There are 4 possible ways that can give rise to a discontinuity in $\Phi(\cdot).$

1. (1)

Job arriving at server $1$ at time $t$.* On arrival of a new job which happens only on server $1$, both $n^{1}(t,q)$, and $n_{o}(t,q)$ increase by $1$ for all $q\in[0,1]$. Hence, there is no change to the $\Phi(t)$ in this case.* 2. (2)

Transfer of jobs between servers under the algorithm (without departure from server $K$) at time $t.$* For each job transitioning from server $i$ to $i+1$ for $i\geq 1$, there is potentially a positive jump in $\Phi(t)$ because of either increase in $n^{i}(t)$ or $n^{i}(t,q)$ for $i\geq 2$. In particular, for $n$ jobs, there are at most $n$ jumps in $\Phi$ for $j=2,\dots,K$, with each jump of size at most*

[TABLE]

since $d^{j}(t,q)\leq A(t)$. Counting for at most $nK$ such jumps, the total increase in $\Phi(t)$ is $\Delta(1)n\sum_{j=2}^{K}c_{j}$. Note that for each jump either $A(t)$ remains same or increases by $1$. In the above bounding we have taken the worst case, when $A(t)$ remains the same. If $A(t)$ increases by $1$, then the same bound follows using second part of Lemma 5.1. Note that transfer of jobs between servers under the $\mathsf{OPT}$ without any departure from server $K$ has no effect on $\Phi(t)$. 3. (3)

Job departing from server $K$ under algorithm at time $t.$* We consider two subcases. If $n^{1}(t)\leq 1,$ then $A(t^{+})=A(t)-1.$ In this case,*

[TABLE]

Here, ($a$) follows from first part of Lemma 5.1, while ($b$) is a consequence of:

[TABLE]

On the other hand, if $n^{1}(t)>1,$ then $A(t^{+})=A(t).$ In this case, it is easy to see that $\Phi_{j}(t^{+})-\Phi_{j}(t)\leq 0.$

Thus, the departure of a job from the system under the algorithm can result in an upward jump in $\Phi(\cdot)$ of at most $\Delta(1)\sum_{j=1}^{K}c_{j}.$ Choosing $c_{j}=c$ for all $j$, the total increase in $\Phi(t)\leq n\Delta(1)\sum_{j=1}^{K}c_{j}$. 4. (4)

Completion of jobs by $\mathsf{OPT}.$* Any job completed by $\mathsf{OPT}$ on server changes $n_{o}(q)$ only for $q=0$ thus, keeping the integral to define $\Phi_{i}(t)$ unchanged for all $i$.*

Lemma 5.1**.**

For $a,d\in\mathbb{N}$ where $d\leq a,$

[TABLE]

Moreover, for $a,d\in\mathbb{N},$

[TABLE]

Proof 5.2**.**

To prove the first statement, we note that

[TABLE]

Here, $(a)$ follows from the monotonicity of $\Delta(\cdot).$ The second statement of the lemma is trivial:

[TABLE]

6. Proof of Lemma 3.9

Our proofs will require the following technical Lemma from **[8]**.

Lemma 6.1**.**

[Lemma 3.1 in [8]] For $s,{\tilde{s}},\beta\geq 0$, then for any function $P$ that is strictly increasing, strictly convex, and differentiable,

[TABLE]

Proof of Lemma 3.9.

Proof 6.2**.**

Since the statement of the lemma applies to a fixed (though generic) time $t,$ we shall omit the reference to $t$ throughout this proof for notational simplicity. Let $q_{i}$ and $q_{o}$ be the size of the job under process at server $i$ with the algorithm, and with the $\mathsf{OPT}$ on server $1$, respectively. Recall that the speed of all active servers with the algorithm is $s_{i}=P^{-1}\left(\frac{n^{1}+A+1}{A+1}\right)$, while the speed of server $1$ with the $\mathsf{OPT}$ is $s_{o}$.

The main idea of bounding $d\Phi/dt$ is similar to [8] being specialized for this potential function and the speed choice.

Case 1: If $n_{o}>n^{1}$, then we first show that $d\Phi_{1}/dt\leq 0$. Note that under this condition, $n_{o}(q)>n^{1}(q)$ for $q\in[q_{o}-s_{o}dt,q_{o}]$. Thus, at time $t+dt$, $n_{o}(q)$ is still at least as much as $n^{1}(q)$ for $q\in[q_{o}-s_{o}dt,q_{o}]$. Therefore, $\Phi_{1}$ does not increase because of processing by $\mathsf{OPT}$. Since processing by the algorithm can only reduce $\Phi_{1}$, thus, $d\Phi_{1}/dt\leq 0$.

Case 2: If $n_{o}<n^{1}$ and $n^{1}>0$ (since otherwise again $d\Phi_{1}/dt=0$). Because of processing of jobs by the algorithm and the $\mathsf{OPT}$, $d\Phi_{1}/dt$ changes because of reduction in $n^{1}(q)$ (because of the algorithm) and $n_{o}(q)$ (because of the $\mathsf{OPT}$). Then for the algorithm, $n^{1}(q)$ decreases by $1$ for $q\in[q^{1}-s^{1}dt,q^{1}]$, and the contribution in $d\Phi_{1}/dt$ because of the algorithm is

[TABLE]

where $\Delta$ has been defined after (7).

Similarly, for the $\mathsf{OPT}$ $n_{o}(q)$ decreases by $1$ for $q\in[q_{o}-s_{o}dt,q_{o}]$, and the contribution in $d\Phi_{1}/dt$ because of the $\mathsf{OPT}$ is

[TABLE]

As shown in [8], that the argument of $\Delta(\cdot)$ is equal in (13) and (14). Combining (13) and (14), we get that

[TABLE]

or

[TABLE]

depending on whether $q_{1}>q_{o}$ or otherwise. For either case, we apply technical Lemma 6.1, to bound RHS of (15) (similar bound will work for (16) as well). Setting $\beta=\frac{n^{1}-n_{o}}{K},$ using Lemma 6.1, note that

[TABLE]

where $(a)$ follows from the speed definition (8) $s_{1}=P^{-1}\left(\frac{n^{1}+A+1}{A+1}\right)$ which ensures that $P(s^{i})\geq\beta$, since $A\leq K-1$. Thus,

[TABLE]

If $n^{1}=n_{o}$ then either $d\Phi_{1}/dt=0$ or (18) applies similar to [8].

Now we bound the $d\Phi_{i}/dt$ for $i\geq 2$, which is easier, since there is no $\mathsf{OPT}$ component in them. Clearly, $d\Phi_{i}/dt=0$ when server $i$ is inactive. So next we consider when server $i$ is active. Let the size of the job being processed by server $i$ with the algorithm be $q^{i}$. Then for the algorithm, $n^{i}(q)$ decreases by $1$ for $q\in[q^{i}-s_{i}dt,q^{i}]$, and the contribution in $d\Phi_{i}/dt$ because of the algorithm is

[TABLE]

where $\Delta$ has been defined after (7). Now, we apply technical Lemma 6.1, to bound RHS of (19). Setting $\beta$ using Lemma 6.1, note that

[TABLE]

where $(a)$ follows from the speed definition (8) which ensures that $P(s^{i})\geq\beta$ (even if $n^{1}=0$ since $\sum_{i=2}^{K}n^{i}\leq A$). Thus, using the Definition (3.8) of $r_{i}$, we get

[TABLE]

which completes the proof.

Remark 6.3**.**

If the set of allowable speeds is bounded, i.e., $[0,\mathsf{B}],$ where $P(\mathsf{B})>1,$ the statement of Theorem 1.1 holds as is. The main points of difference are in the proof of Lemma 3.9, in the two applications of Lemma 6.1. The first application (see (17)) holds due to the justification in [8]. Since $\beta$, the second application (see (LABEL:eq:dummy457)) requires $P(s^{i})\geq\beta$ which holds so long as $P(\mathsf{B})>1.$

7. Proof of Theorem 4.1

Let $C_{i}=\mathbb{E}\left[T_{i}\right]+\mathbb{E}\left[E_{i}\right]$ denote the cost associated with layer $i.$ We prove the Theorem via deriving two lower bounds as follows.

Lemma 7.1**.**

For any policy,

[TABLE]

Proof 7.2**.**

This lower bound is obtained using considering only the energy cost. Let $S_{i,j}$ denote the steady state speed of server $j$ in layer $i.$

[TABLE]

Here, $(a)$ follows by applying Jensen's inequality. $(b)$ is a consequence of the convexity of $P(\cdot)$ along with $\sum_{j=1}^{m_{i}}\mathbb{E}\left[S_{i,j}\right]=\rho_{i}.$

Lemma 7.3**.**

For any policy,

[TABLE]

Proof 7.4**.**

This lower bound comes from optimizing the cost of serving a single job in isolation. Indeed,

[TABLE]

The first term above is ($\lambda$ times) the delay cost, and the second is ($\lambda$ times) the energy cost. The above optimization can be solved explicitly, yielding the statement of the lemma.

We are now ready to prove Theorem 4.1. Let $C_{i}^{A}$ denote the cost associated with the proposed algorithm. This is given by

[TABLE]

The above expressions follow since layer $i$ receives arrivals as per a Poisson process (this is a consequence of Burke's theorem **[18]**), which is further split into $m_{i}$ independent Poisson streams feeding into the $m_{i}$ servers in layer $i.$ Indeed, the steady state mean response time in layer $i$ equals $\frac{1}{\mu_{i}(s_{i}^{A}-\rho_{i}/m_{i})},$ and $\lambda$ times the energy per job equals the stationary power consumption, given by the second term in (21).

Following (21) and Lemma 7.1 and Lemma 7.3, we get

[TABLE]

Finally, we can bound the overall cost of the proposed algorithm as follows.

[TABLE]