Multi-tier Caching Analysis in CDN-based Over-the-top Video Streaming Systems

TL;DR

This paper models multi-tier CDN caching for video streaming, analyzing various design factors to minimize stall duration tail probability, and proposes an efficient algorithm validated through cloud system experiments.

Contribution

It introduces a comprehensive model for multi-tier CDN caching in video streaming and develops a novel algorithm to optimize performance metrics considering multiple system design factors.

Findings

The proposed algorithm significantly reduces stall duration tail probability.

Theoretical bounds for SDTP are established.

Implementation results show improved QoE in cloud environments.

Abstract

Internet video traffic has been been rapidly increasing and is further expected to increase with the emerging 5G applications such as higher definition videos, IoT and augmented/virtual reality applications. As end-users consume video in massive amounts and in an increasing number of ways, the content distribution network (CDN) should be efficiently managed to improve the system efficiency. The streaming service can include multiple caching tiers, at the distributed servers and the edge routers, and efficient content management at these locations affect the quality of experience (QoE) of the end users. In this paper, we propose a model for video streaming systems, typically composed of a centralized origin server, several CDN sites, and edge-caches located closer to the end user. We comprehensively consider different systems design factors including the limited caching space at the CDN sites, allocation of CDN for a video request, choice of different ports (or paths) from the CDN and the central storage, bandwidth allocation, the edge-cache capacity, and the caching policy. We focus on minimizing a performance metric, stall duration tail probability (SDTP), and present a novel and efficient algorithm accounting for the multiple design flexibilities. The theoretical bounds with respect to the SDTP metric are also analyzed and presented. The implementation on a virtualized cloud system managed by Openstack demonstrate that the proposed algorithms can significantly improve the SDTP metric, compared to the baseline strategies.

Figures

Tables

Table 1. TABLE I : The value of α j subscript 𝛼 𝑗 \alpha_{j} used in the evaluation results with units of 1/ms. We set η j , β j ( d ) = η j , β j ( d ¯ ) = η j , ν j ( e ) = 14 superscript subscript 𝜂 𝑗 subscript 𝛽 𝑗 𝑑 superscript subscript 𝜂 𝑗 subscript 𝛽 𝑗 ¯ 𝑑 superscript subscript 𝜂 𝑗 subscript 𝜈 𝑗 𝑒 14 \eta_{j,\beta_{j}}^{(d)}=\eta_{j,\beta_{j}}^{(\overline{d})}=\eta_{j,\nu_{j}}^{(e)}=14 ms.

Node 1	Node 2	Node 3	Node 4	Node 5	Node 6
$82.00$	$76.53$	$71.06$	$65.6$	$60.13$	$54.66$

Table 2. TABLE II : Testbed Configuration

Cluster Information
Control Plane	Openstack Kilo
VM flavor	1 VCPU, 2GB RAM, 20G storage (HDD)
Software Configuration
Operating System	Ubuntu Server 16.04 LTS
Origin Server(s)	Apache Web Server [36]: Apache/2.4.18 (Ubuntu)
Cache Server(s)	Apace Traffic Server [37] 6.2.0 (build # 100621)
Client	Apache JMeter [38] with HLS plugin [39]

Table 3. TABLE III: Key Notations Used in This Paper

Symbol	Meaning
$r$	Number of video files in system
$m$	Number of storage nodes
$L_i,L_{j,i}$	Number of video segments and cached chunks for video file $i$, respectively, at server $j$
${\lambda }_i$	Possion arrival rate of video file $i$
${\pi }_{i,j,\ell }$	Probability of retrieving chunk of file $i$ from node $j$ from edge-router $\ell$ using probabilistic scheduling algorithm
$p_{i,j,{\nu }_j\ell }$	Probability of retrieving chunk of file $i$ from server $j$ and PS ${\nu }_j$, if requested through edge-router $\ell$.
$q_{i,j,{\beta }_j\ell }$	Probability of retrieving chunk of file $i$ from server $j$ and PS ${\beta }_j$, if requested through edge-router $\ell$.
${\text{PS}}_{{\beta }_j,\ell }^{(d,j)},\forall {\beta }_j$	Set of parallel streams between data center and cache server $j$ which serves edge router $\ell$
${\text{PS}}_{{\beta }_j,\ell }^{(\overline{d},j)},\forall {\beta }_j$	Set of parallel streams between cache server $j$ and edge-router $\ell$
${\text{PS}}_{{\nu }_j,\ell }^{(e,j)},\forall {\nu }_j$	Set of parallel streams between cache server $j$ and edge-router $\ell$ assigned to serve cached segments
$({\alpha }_j^{\left(d\right)},{\eta }_j^{\left(d\right)})$	Parameters of Shifted Exponential distribution of service time from data center to cache server $j$
$({\alpha }_{j,\ell }^{\left(f_j\right)},{\eta }_{j,\ell }^{\left(f_j\right)})$	Parameters of Shifted Exponential distribution of service time from cache server $j$ to edge-router $\ell$
$M_{j,{\beta }_j,\ell }^{\left(d\right)}$	Moment generating function for the service time of the parallel stream ${\text{PS}}_{{\beta }_j,\ell }^{(d,j)}$
$M_{j,{\beta }_j,\ell }^{\left(\overline{d}\right)}$	Moment generating function for the service time of the parallel stream ${\text{PS}}_{{\beta }_j,\ell }^{(\overline{d},j)}$
$M_{j,{\beta }_j,\ell }^{\left(e\right)}$	Moment generating function for the service time of the parallel stream ${\text{PS}}_{{\nu }_j,\ell }^{(e,j)}$
${\Lambda }_{j,{\beta }_j,\ell }^{\left(d\right)}$	aggregate arrival rate at parallel stream ${\text{PS}}_{{\beta }_j,\ell }^{(d,j)}$
${\Lambda }_{j,{\beta }_j,\ell }^{\left(\overline{d}\right)}$	aggregate arrival rate at parallel stream ${\text{PS}}_{{\beta }_j,\ell }^{(\overline{d},j)}$
${\Lambda }_{j,{\beta }_j,\ell }^{(e,j)}$	aggregate arrival rate at parallel stream ${\text{PS}}_{{\beta }_j,\ell }^{(e,j)}$
$\sigma$	Parameter indexing stall duration tail probability
$X_{i,\ell }$	Random variable corresponding to amount of space in the edge-cache $\ell$ for video file $i$
$T_{i,j,{\beta }_j,{\nu }_j}^{\left(u\right)}$	Time that chunk $u$ begins to play at client $i$ given that it is downloaded from ${\beta }_j$ and ${\nu }_j$.
${\Gamma }_U^{i,j,{\beta }_j,{\nu }_j}$	Stall duration for the request of file $i$ from ${\beta }_j$ and ${\nu }_j$ queues (not cached at the edge-router).
${\Gamma }_{tot}^{i,j,{\beta }_j,{\nu }_j}$	Total stall duration for the request of file $i$ from either ${\beta }_j$ and ${\nu }_j$ queues or from the edge-cache.
$D_{i,j,{\beta }_j,{\nu }_j}^{\left(u\right)}$	Download time fo chunk $u\in \{1,\dots ,L_i\}$ of file $i$ from node $j$, from ${\beta }_j$ and ${\nu }_j$ queues.
$R_{j,{\nu }_j}^{\left(e\right)}$	Service time of the video files at the parallel streams ${\text{PS}}_{\left({\nu }_j\right)}^{(e,j)}$
$B_{j,{\beta }_j}^{\left(d\right)}$	MGF of the Service time of all video files from the parallel stream ${\text{PS}}_{\left({\beta }_j\right)}^{(d,j)}$
${\rho }_{j,{\nu }_j}^{\left(e\right)}$	load intensity at the parallel stream ${\text{PS}}_{\left({\nu }_j\right)}^{(e,j)}$.
${\rho }_{j,{\beta }_j}^{\left(d\right)}$	load intensity at the parallel stream ${\text{PS}}_{\left({\beta }_j\right)}^{(d,j)}$.
${\rho }_{j,{\beta }_j}^{\left(\overline{d}\right)}$	load intensity at the parallel stream ${\text{PS}}_{\left({\beta }_j\right)}^{(\overline{d},j)}$.
$d_s$	Start-up delay
$\tau$	Chunk size in seconds
$\theta$	Trade off factor between mean stall duration and stall duration tail probability

Equations

$$\sum _{{\beta }_j=1}^{d_j}{w}_{j,{\beta }_j,\ell }^{(d)}\le 1,\sum _{{\beta }_j=1}^{d_j}{w}_{j,{\beta }_j,\ell }^{(\overline{d})}+\sum _{{\nu }_j=1}^{e_j^{(\ell )}}{w}_{j,{\nu }_j,\ell }^{(e)}\le 1,$$

$${\alpha }_{j,{\beta }_j,\ell }^{(d)}=w_{j,{\beta }_j,\ell }^{(d)}{\alpha }_j^{(d)},$$

$${\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}=w_{j,{\beta }_j,\ell }^{(\overline{d})}{\alpha }_{j,\ell }^{(f_j)},{\alpha }_{j,{\nu }_j,\ell }^{(e)}=w_{j,{\nu }_j,\ell }^{(e)}{\alpha }_j^{(f_j)},$$

$$M_{j,{\beta }_j,\ell }^{(d)}=\frac{{\alpha }_{j,{\beta }_j,\ell }^{(d)}{e}^{{\eta }_{j,{\beta }_j,\ell }^{(d)}t}}{{\alpha }_{j,{\beta }_j,\ell }^{(d)}-t},$$

$$M_{j,{\beta }_j,\ell }^{(\overline{d})}=\frac{{\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}{e}^{{\eta }_{j,{\beta }_j,\ell }^{(\overline{d})}t}}{{\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}-t},$$

$$M_{j,{\nu }_j,\ell }^{(e)}=\frac{{\alpha }_{j,{\nu }_j,\ell }^{(e)}{e}^{{\eta }_{j,{\nu }_j,\ell }^{(e)}t}}{{\alpha }_{j,{\nu }_j,\ell }^{(e)}-t}\text{vspace}-.4in$$

$$\sum _{j=1}^m{\pi }_{i,j,\ell }=1\forall i,\ell ;$$

$$\sum _{{\nu }_j=1}^{e_j^{(\ell )}}{p}_{i,j,{\nu }_j,\ell }=1\forall i,j,\ell ;$$

$$\sum _{{\beta }_j=1}^{d_j}{q}_{i,j,{\beta }_j,\ell }=1\forall i,j,\ell ,$$

$${\pi }_{i,j,\ell },p_{i,j,{\nu }_j,\ell },q_{i,j,{\beta }_j,\ell }\ge 0\forall i,j,{\beta }_j,{\nu }_j,\ell$$

$${\Lambda }_{j,{\beta }_j,\ell }^{(d)}=\sum _{i=1}^r{\lambda }_{i,\ell }{\pi }_{i,j,\ell }{q}_{i,j,{\beta }_j,\ell }{e}^{-{\lambda }_{i,\ell }{\omega }_{i,\ell }};{\Lambda }_{j,{\beta }_j,\ell }^{(\overline{d})}={\Lambda }_{j,{\beta }_j,\ell }^{(d)},$$

$${\Lambda }_{j,{\nu }_j,\ell }^{(e)}=\sum _{i=1}^r{\lambda }_{i,\ell }{\pi }_{i,j,\ell }{p}_{i,j,{\nu }_j,\ell }{e}^{-{\lambda }_{i,\ell }{\omega }_i}$$

$$P({\tilde{t}}_i>{\omega }_i)=\frac{{\nu }_i}{{\nu }_i+{\lambda }_i}$$

$$X_{i,\ell }=\{\begin{array}{ll}\tau L_i& \text{with prob. 1−e−λi,ℓωi,ℓ}\\ 0& \text{with prob. }{e}^{-{\lambda }_{i,\ell }{\omega }_{i,\ell }}\end{array}$$

$$X_{\ell }=\sum _{i=1}^r{X}_{i,\ell }$$

$$\underset{C_{\ell ,e}}{\overset{\infty }{\int }}\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}dx\le {ϵ}_{\ell }$$

$${\Gamma }_U^{(i,j,{\beta }_j,{\nu }_j)}=T_{i,j,{\beta }_j,{\nu }_j}^{(L_i)}-d_s-\left(L_i-1\right)\tau ,$$

$${\Gamma }_{tot}^{(i)}\stackrel{d}{=}\{\begin{array}{ll}{\left({\Gamma }_{tot}^{(i)}-{\tilde{t}}_i\right)}^{+}& 0\le {\tilde{t}}_i\le {\omega }_i\\ {\Gamma }_U^{(i,j,{\beta }_j,{\nu }_j)}& {\tilde{t}}_i>{\omega }_i\end{array}$$

$$\mathbb{E}\left[e^{h_i{\Gamma }_{tot}^{(i)}}\right]$$

$$\mathbb{E}\left[e^{h_i{D}_{i,j,{\beta }_j,{\nu }_j}^{(v)}}|(j,{\beta }_j,{\nu }_j)\right]$$

$$\mathbb{E}\left[e^{h_i{D}_r^{(v)}}|(j,{\beta }_j,{\nu }_j)\right]$$

$$\text{Pr}\left({\Gamma }_{tot}^{(i,\ell )}\ge \sigma \right)\le$$

$$\displaystyle\sum_{j=1}^{m}\pi_{i,j,\ell}\times\Biggl{[}\overline{c}_{\ell}+\widetilde{a}_{\ell}\,e^{-h_{i}\sigma}+\overline{a}_{\ell}\sum_{\nu_{j}=1}^{e_{j}^{(\ell)}}p_{i,j,\nu_{j},\ell}\times$$

$$\sum _{{\beta }_j=1}^{d_j}{q}_{i,j,{\beta }_j,\ell }{e}^{h_i{L}_i\tau }\times \left({\delta }^{(e,\ell )}+{\delta }^{(\overline{d},\ell )}+{\delta }^{(d,\overline{d},\ell )}\right)$$

$${\delta }^{(e,\ell )}$$

$${\delta }^{(\overline{d},\ell )}$$

$${\delta }^{(d,\overline{d},\ell )}$$

$${\xi }_{i,j,{\beta }_j,}^{(d,\overline{d},\ell )}$$

$${\gamma }^{(d,\ell )}$$

$${\tilde{M}}_{j,{\beta }_j}^{(d)}(h_i)=\frac{{\alpha }_{j,{\beta }_j}^{(d)}{e}^{{\eta }_{j,{\beta }_j}-h_i\tau }}{{\alpha }_{j,{\beta }_j}^{(d)}-h_i},$$

$${\tilde{M}}_{j,{\beta }_j}^{(\overline{d},\ell )}(h_i)=\frac{{\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}{e}^{{\eta }_{j,{\beta }_j,\ell }-h_i\tau }}{{\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}-h_i}$$

$${\tilde{M}}_{j,{\nu }_j}^{(e,\ell )}(h_i)=\frac{{\alpha }_{j,{\nu }_j,\ell }^{(e)}{e}^{{\eta }_{j,{\nu }_j,\ell }-h_i\tau }}{{\alpha }_{j,{\nu }_j,\ell }^{(e)}-h_i},$$

$${\tilde{M}}_{j,{\beta }_j,\ell }^{(d,\overline{d})}(h)=\frac{{\alpha }_{j,{\beta }_j}^{(d)}({\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}-h)e^{{\eta }_{j,{\beta }_j}^{(d)}h}}{{\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}({\alpha }_{j,{\beta }_j}^{(d)}-h)e^{{\eta }_{j,{\beta }_j,\ell }^{(\overline{d})}h}},\forall j,{\beta }_j$$

$${\overline{c}}_{\ell }=\frac{(1-e^{-{\lambda }_{i,\ell }{\omega }_{i,\ell }})e^{-h_i\sigma }}{1-\frac{{\lambda }_i}{{\lambda }_i+h_i}\left(1-e^{-\left({\lambda }_i+h_i\right){\omega }_i}\right)}$$

$${\tilde{a}}_{\ell }=\frac{e^{-{\lambda }_{i,\ell }{\omega }_{i,\ell }}}{1-\frac{{\lambda }_i}{{\lambda }_i+h_i}\left(1-e^{-\left({\lambda }_i+h_i\right){\omega }_i}\right)}$$

$${\overline{a}}_{\ell }={\tilde{a}}_{\ell }{e}^{-h_i(\sigma +d_s+(L_i-1)\tau )}$$

$$\displaystyle\sum_{\ell=1}^{R}\sum_{i=1}^{r}\frac{\lambda_{i,\ell}}{\overline{\lambda}_{i,\ell}}\sum_{j=1}^{m}\pi_{i,j,\ell}\times\Biggl{[}\overline{c}_{\ell}+\widetilde{a}_{\ell}\,e^{-h_{i}\sigma}+\overline{a}_{\ell}\sum_{\nu_{j}=1}^{e_{j}}p_{i,j,\nu_{j},\ell}\times$$

$$\sum _{{\beta }_j=1}^{d_j}{q}_{i,j,{\beta }_j,\ell }{e}^{h_i{L}_i\tau }\times \left({\delta }^{(e)}+{\delta }^{(\overline{d})}+{\delta }^{(d,\overline{d})}\right)$$

$$\text{eqref}sumWs-\text{eqref}lambda{e}_{n}uj,\text{eqref}edgeCacheConst,\text{eqref}deltae-\text{eqref}eq:abar$$

$${\rho }_{j,{\beta }_j,\ell }^{(d)}=\sum _{i=1}^r{\lambda }_{i,\ell }{\pi }_{i,j}{q}_{i,j,{\beta }_j,\ell }{e}^{{\lambda }_{i,\ell }{\omega }_{i,\ell }}\frac{L_i-L_{j,i}}{{\alpha }_{j,{\beta }_j,\ell }^{(d)}}<1,\forall j,{\beta }_j$$

$${\rho }_{j,{\beta }_j,\ell }^{(\overline{d})}=\sum _{i=1}^r{\lambda }_{i,\ell }{\pi }_{i,j,\ell }{q}_{i,j,{\beta }_j,\ell }{e}^{{\lambda }_{i,\ell }{\omega }_{i,\ell }}\frac{L_i-L_{j,i}}{{\alpha }_{j,{\beta }_j,\ell }^{(\overline{d})}}<1\forall j,{\beta }_j,\ell$$

Full text

In this paper, we propose a model for video streaming systems, typically composed of a centralized origin server, several CDN sites, and edge-caches located closer to the end user. We comprehensively consider different systems design factors including the limited caching space at the CDN sites, allocation of CDN for a video request, choice of different ports (or paths) from the CDN and the central storage, bandwidth allocation, the edge-cache capacity, and the caching policy. We focus on minimizing a performance metric, stall duration tail probability (SDTP), and present a novel and efficient algorithm accounting for the multiple design flexibilities. The theoretical bounds with respect to the SDTP metric are also analyzed and presented. The implementation on a virtualized cloud system managed by Openstack demonstrate that the proposed algorithms can significantly improve the SDTP metric, compared to the baseline strategies.

I Introduction

Over-the-top video streaming, e.g., Netflix and YouTube, has been dominating the global IP traffic in recent years. The traffic will continue to grow due to the introduction of even higher resolution video formats such as 4K on the horizon. As end-users consume video in massive amounts and in an increasing number of ways, service providers need flexible solutions in place to ensure that they can deliver content quickly and easily regardless of their customer’s device or location. More than $50\%$ of over-the-top video traffic are now delivered through content distribution networks (CDNs) [2]. Even though multiple solutions have been proposed for improving congestion in the CDN system, managing the ever-increasing traffic requires a fundamental understanding of the system and the different design flexibilities (control knobs) to make the best use of the limited hardware resources. This is the focus of this paper.

The service providers typically use two-tier caching approach to improve the quality of streaming services [3, 4, 5]. In addition to the distributed cache servers provided by the CDN, the edge router can also have a cache so that some videos could be stored in this cache and gets the advantage of the proximity to end-users. However, there are many edge routers which imply that the hot content could be stored at multiple edge routers. There is an additional cache at the distributed cache servers (in CDN) from which data can be obtained if not already at the edge router. Such multi-tier caching is related to fog computing where the caching could be distributed at multiple locations in the network [4]. We also assume that the edge cache can help provide advantages similar to multicasting. If another user on the edge router is already consuming the file, the part of the video already downloaded is sent directly to the new user and the later part is sent to multiple users who requested the content on the same edge router. This paper aims to analyze two-tiered caching in video streaming systems.

In this paper, we consider an architecture of streaming system with a Virtualized Content Distribution Network (vCDN) [6, 7]. The main role of this CDN infrastructure is not only to provide users with lower response time and higher bandwidth, but also to distribute the load (especially during peak time) across many edge locations. The infrastructure consists of a remote datacenter that stores complete original video data and multiple CDN sites (i.e., distributed cache servers) that only have part of those data and are equipped with solid state drives (SSDs) for high throughput. In addition, we assume that a second caching tier is located at the edge routers. A user request for video content not served from the edge cache is directed to a distributed cache. If it is still not completely served, the remaining part of the request is directed to the remote datacenter (as shown in Fig. 1). Multiple parallel connections are established between the distributed cache server and the edge router, as well as between the distributed cache servers and the origin server, to support multiple video streams simultaneously. Our goal is to develop an optimization framework and QoE metrics that service providers (or infrastructure) could use to answer the following questions: How to quantify the impact of multi-tier video caching on end user experience? What is the best video multi-tier caching strategy for CDN? How to optimize QoE metrics over various “control knobs"? Are there enough benefits to justify the adoption of proposed solutions in practice?

It has been shown that, in modern cloud applications such as Facebook, Bing, and Amazon’s retail platform, the long tail of latency is of a major concern, with $99.9$th percentile response times that are, orders of magnitude worse than the mean [8, 9]. Thus, this paper considers a QoE metric, called the stall duration tail probability (SDTP), which measures the likelihood of end users suffering a worse-than-expected stall duration, and develop a holistic optimization framework for minimizing the overall SDTP over joint caching content placement, network resource optimization and user request scheduling. SDTP, denoted by Pr$(\Gamma^{(i)}>\sigma)$, measures the probability that the stall duration $\Gamma^{(i)}$ of video $i$ is greater than a pre-defined threshold $\sigma$. Despite resource and load-balancing mechanisms, large scale storage systems evaluations show that there is a high degree of randomness in delay performance [10]. In contrast to web object caching and delivery, the video chunks in the latter part of a video do not have to be downloaded much earlier than their actual play time to maintain the desired QoE, making SDTP highly dependent on the joint optimization with resource management and request scheduling in CDN-based video streaming.

Quantifying SDTP with multi-tier cache/storage is an open problem. Even for single-chunk video files, the problem is equivalent to minimizing the download tail latency, which is still an open problem [11]. The key challenge arises from the difficulty of constructing and analyzing a scheduling policy that (optimally) redirects each request based on dependent system and queueing dynamics (including cache content, network conditions, request queue status) on the fly. To overcome these challenges, we propose a novel two-stage, probabilistic scheduling approach, where each request of video $i$ is (i) processed by cache server $j$ with probability $\pi_{i,j}$ and (ii) assigned to video stream $v$ with probability $p_{i,j,v}$. The two-stage, probability scheduling allows us to model each cache server and video stream as separate queues, and thus, to characterize the distributions of different video chunks’ download time and playback time. Further, the edge caching policy plays a key role in the system design. This paper proposes an adaption of least-recently-used (LRU) caching mechanism [12], where each file is removed from the edge cache if it has not been requested again for a time that depends on the edge router and the file index. By optimizing these probabilities and the edge-cache parameters, we quantify SDTP through a closed-form, tight upper bound for CDN-based video streaming with arbitrary cache content placement and network resource allocation. We note that the analysis in this paper is fundamentally different from those for distributed file storage, e.g., [13, 14], because the stall duration of a video relies on the download times of all its chunks, rather than simply the time to download the last chunk of a file. Further, since video chunks are downloaded and played sequentially, the download times and playback times of different video chunks are highly correlated and thus jointly determine the SDTP metric.

This paper proposes a holistic optimization framework for minimizing overall SDTP in CDN-based video streaming. To the best of our knowledge, this is the first framework for multi-tier caching to jointly consider all key design degrees of freedom, including bandwidth allocation among different parallel streams, multi-tier cache content placement and update, request scheduling, and the modeling variables associated with the SDTP bound. An efficient algorithm is then proposed to solve this non-convex optimization problem. In particular, the proposed algorithm performs an alternating optimization over the different dimensions, such that each sub-problem is shown to have convex constraints and thus can be efficiently solved using the iNner cOnVex Approximation (NOVA) algorithm proposed in [15]. The proposed algorithm is implemented in a virtualized cloud system managed by Openstack [16]. The experimental results demonstrate significant improvement of QoE metrics as compared to the considered baselines.

The main contributions of this paper can be summarized as follows:

• •

We propose a novel framework for analyzing CDN-based over-the top video streaming systems with the use of multiple caching tiers and multiple parallel streams between nodes. A novel two-stage probabilistic scheduling policy is proposed to assign each user request to different cache servers and parallel video streams. Further, the edge router uses an adaptation of LRU, and the distributed cache servers cache partial files.

• •

The distribution of (random) download time of different video chunks are analyzed. Then, using ordered statistics, we quantify the playback time of each video segment.

• •

Multiple recursive relations are set up to compute the stall duration tail probability. We first relate the compute of download time of each chunk to the play time of each chunk, Since the play time depends not only on download of the current chunk, but also on the previous chunks. Second, stall duration must account for whether the file has been requested by anyone within a window time of a certain size to get advantage of edge cache. If it had been requested, the stall duration is a function of the time of the last request and the stall duration at that time. In the steady state analysis, this will lead to a recursion. This analysis has been used to derive an analytical upper bound on SDTP for arbitrary distributed cache content placement, parameters of edge cache, and the parameters of the two-stage probabilistic scheduling (Appendix B).

• •

A holistic optimization framework is developed to optimize a weighted sum of SDTP of all video files over the request scheduling probabilities, distributed cache content placement, the bandwidth allocation among different streams, edge cache parameters, and the modeling parameters in SDTP bound. An efficient algorithm is provided to decouple and solve this non-convex optimization (Section V).

• •

To better understand the SDTP and how it relates to the QoE of users, we correlate this metric to a well-known QoE metric (called mean stall duration). Since the optimal point for the mean stall duration is not the same as that of the SDTP, we optimize a convex combination of the two metrics and show how the two QoE metrics can be compromised based on the point on the curve that is appropriate for the clients (Appendix I).

• •

The algorithm is implemented on a virtualized cloud system managed by Openstack. The simulation and trace-based results validate our theoretical analysis with the implementation and analytical results being close, thus demonstrating the efficacy of our proposed algorithm. The QoE metric is shown to have significant improvement as compared to competitive strategies (Appendix VI).

The rest of this paper is organized as follows. Section II provides related work for this paper. In Section III, we describe the system model used in the paper with a description of CDN-based Over-the-top video streaming systems. Section IV provides an upper bound on the mean stall duration. Section V formulates the QoE optimization problem as a weighted sum of all SDTP of all files and proposes the iterative algorithmic solution of this problem. Experimental results are provided in Section VI. Section VII concludes the paper.

II Related Work

Video on Demand services and Live TV Content from cloud servers have been studied widely [17, 18, 19, 20, 21]. The placement of content and resource optimization over the cloud servers have been considered. In [20], authors utilize the social information propagation pattern to improve the efficiency of social video distribution. Further, they used replication and user request dispatching mechanism in the cloud content delivery network architecture to reduce the system operational cost, while maintaining the averaged service latency. However, [20] only considers video download. The benefits of delivering videos at the edge network is shown in [21]. Authors show that bringing videos at the edge network can significantly improve the content item delivery performance, in terms of improving quality experienced by users as well as reducing content item delivery costs. To the best of our knowledge, reliability of content over the cloud servers have not been considered for video streaming applications. There are novel challenges to characterize and optimize the QoE metrics at the end user. Adaptive streaming algorithms have also been considered for video streaming [22, 23, 24] [25] which are beyond the scope of this paper and are left for future work.

Mean latency and tail latency have been characterized in [13, 14, 26] and [27, 28], respectively, for a system with multiple files using probabilistic scheduling. However, these papers consider only file downloading rather than video streaming. This paper considers CDN-based video streaming. We note that file downloading can follow as a special case of streaming, which makes our model more general. Additionally, the metrics for video streaming do not only account for the end of the download of the video but also for the download of each segment. Hence, the analysis for the content download cannot be extended to the video streaming directly and the analysis approach in this paper is very different from the prior work in the area of file downloading.

More recently, the authors of [29] considered video-streaming over distributed storage systems. However, caching placement optimization and is not considered. Also, caching at the edge level is not considered. Moreover, only a single stream between each storage server and edge node is assumed and hence neither the two-stage probabilistic scheduling nor bandwidth allocation were considered. Similarly, there is no edge cache in [30]. Thus, the analysis and the problem formulation here is an extension of that in [29, 30].

III System Model

III-A Target System

Our work is motivated by the architecture of a production system with a Virtualized Content Distribution Network (vCDN). Such services, for instance, include video-on-demand (VoD), live linear streaming services (also referred to as over-the-top video streaming services), firmware over the air (FOTA) Android updates to mobile devices, etc. The main role of this CDN infrastructure is not only to provide users with lower response time and higher bandwidth, but also to distribute the load (especially during peak time) across many edge locations. Consequently, the core backbone network will have reduced network load and better response time. The origin server has original data and CDN sites have only part of those data. Each CDN site is composed of multiple cache servers each of which is typically implemented as a VM backed by multiple directly attached solid state drives (SSDs) for higher throughput. The cache servers store video segments and a typical duration of each segment covers $5-11$ seconds of playback time.

Further, the typical vCDN architecture includes an additional cache at the edge, called edge cache. This edge cache allows for saving some recently accessed videos. This cache can also help multicasting content to another user connected to the same edge router. One of the typical policy that is used in edge cache is based on least-recently-used (LRU) caching policy [12]. In this paper, we will consider a modification of this strategy to weigh the eviction policy of contents dependent on their weight, placement, and access rates and thus can be optimized.

When a client such as VoD/LiveTV app requests a certain content, it goes through multiple steps. First, it sees whether the content is in edge cache. If so, the content is directly accessed from the edge cache. Then, it sees whether the content has been requested by someone connected to the same edge router and is being sent to them. In this case, the content already received at the edge router is sent to the user and the remaining content is passed as received (equivalent to a multicast stream setup). If the content cannot be obtained in the two steps, the client then contacts CDN manager, choose the best CDN service to use and retrieve a fully qualified domain name (FQDN). Fourth, with the acquired FQDN, it gets a cache server’s IP address from a content routing service (called iDNS). Then we use the IP address to connect to one of the cache servers. The cache server will directly serve the incoming request if it has data in its local storage (cache-hit). If the requested content is not on the cache server (i.e., cache-miss), the cache server will fetch the content from the origin server and then serve the client.

In the rest of this section, we will present a generic mathematical model applicable to not only our vCDN system but also other video streaming systems that implement CDN-like two-tier caching structure.

III-B System Description

We consider a content delivery network as shown in Fig. 1, consisting of a single datacenter that has an origin server, $m$ geographically-distributed cache servers denoted by $j=1,\ldots,m$, and edge-cache storage nodes associated with the edge routers $\ell\in\{1,2,\cdots,R\}$, where $R$ is the total number of edge-routers, as depicted in Figure 2. The compute cache servers (also called storage nodes) are located close to the edge of the network and thus provide lower access latency for end users. We also assume that each cache server is connected to one edge router. Further, the connection from the edge router to the users is not considered as a bottleneck. Thus, the edge router is considered as a combination of users and is the last hop for our analysis. We also note that the link from the edge router to the end users is not controlled by the service provider and thus cannot be considered for optimized resource allocation from the network. The service provider wishes to optimize the links it controls for efficient quality of experience to the end user.

A set of $r$ video files (denoted by $i=1,\ldots,r$) are stored in the datacenter, where video file $i$ is divided into $L_{i}$ equal-size segments each of length $\tau$ seconds. We assume that the first $L_{j,i}$ chunks of video $i$ are stored on cache server $j$. Even though we consider a fixed cache placement, we note that $L_{j,i}$ are optimization variables and can be updated when sufficient arrival rate change is detected.

We assume that the bandwidth between the data center and the cache server $j$ (service edge router $\ell$) is split into $d_{j}$ parallel streams, where the streams are denoted as $PS^{(d,j)}_{\beta_{j},\ell}$ for $\beta_{j}=1,\cdots,d_{j}$. Further, the bandwidth between the cache server $j$ and the edge router $\ell$ is divided into $f_{j}^{(\ell)}$ parallel streams, denoted as $PS^{(f,j)}_{\zeta_{j},\ell}$ for $\zeta_{j}^{(\ell)}=1,\cdots,f_{j}^{(\ell)}$, and $\ell=1,2,\cdots,R$. Multiple parallel streams are assumed for video streaming since multiple video downloads can happen simultaneously. Since we care about stall duration, obtaining multiple videos simultaneously is helpful as the stall durations of multiple videos can be improved. We further assume that $f_{j}^{(\ell)}$ PSs are divided into two set of streams $d_{j}$ and $e_{j}^{(\ell)}$. This setup is captured in Figure 2. The first $d_{j}$ parallel streams are denoted as $PS^{(\overline{d},j)}_{\beta_{j},\ell}$ for $\beta_{j}=1,\cdots,d_{j}$, and for all $\ell$ while the remaining $e_{j}^{(\ell)}$ streams are denoted as $PS^{(e,j)}_{\nu_{j},\ell}$ for $\nu_{j}=1,\cdots,e_{j}^{(\ell)}$. In order to consider the splits, $PS^{(d,j)}_{\beta_{j},\ell}$ gets $\{w_{j,\beta_{j},\ell}^{(d)},\,\beta_{j}=1,\ldots,d_{j}\}$ fraction of the bandwidth from the data center to the cache server $j$. Similarly, $PS^{(\overline{d},j)}_{\beta_{j},\ell}$ gets $\{w_{j,\beta_{j},\ell}^{(\overline{d})},\,\beta_{j}=1,\ldots,d_{j}\}$ fraction of bandwidth from cache server $j$ to the edge router $\ell$ and $PS^{(e,j)}_{\nu_{j},\ell}$ gets $\{w_{j,\nu_{j},\ell}^{(e)},\,\nu_{j}=1,\ldots,e_{j}^{(\ell)}\}$ fraction of bandwidth from cache server $j$ to the edge router $\ell$. Thus, we have

[TABLE]

for all $j=1,\cdots,m$ and $\ell=1,\cdots,R$. We note that the sum of weights may be less than $1$ and some amount of the bandwidth may be wasted. While the optimal solution will satisfy this with equality since for better utilization, we do not need to explicitly enforce the equality constraint. We note that if the cache server serves multiple edge routers, the parallel streams between cloud storage and cache server will be the sum of $d_{j}$ to each edge router thus making the problem separated for each edge router. For ease, we will sometime omit $\ell$ to focus on links to one edge router only and the same procedure can be used for each edge router.

We assume that the service time of a segment for data transfer from the data center to the cache server $j$ is shifted-exponential with rate $\alpha_{j,\ell}^{(d)}$ and a shift of $\eta_{j,\ell}^{(d)}$ while that between the cache server $j$ and the edge router $\ell$ is also shifted-exponential with rate $\alpha_{j,\ell}^{(f_{j})}$ and a shift of $\eta_{j,\ell}^{(f_{j})}$. The shifted exponential distribution can be seen as an approximation of the realistic service time distribution in the prior works, e.g., [31], and references therein. We also note that the rate of a parallel stream is proportional to the bandwidth split. Thus, the service time distribution of $PS^{(d,j)}_{\beta_{j},\ell}$, $PS^{(\overline{d},j)}_{\beta_{j},\ell}$, and $PS^{(e,j)}_{\nu_{j},\ell}$, denoted as $\alpha_{j,\beta_{j},\ell}^{(d)}$, $\alpha_{j,\beta_{j},\ell}^{(\overline{d})}$, and $\alpha_{j,\nu_{j},\ell}^{(\overline{d})}$, respectively, and are given as follows.

[TABLE]

for all $\beta_{j}$, $\nu_{j}$, and $\ell$. We further define the moment generating functions of the service times of $PS^{(d,j)}_{\beta_{j},\ell}$, $PS^{(\overline{d},j)}_{\beta_{j},\ell}$, and $PS^{(e,j)}_{\nu_{j},\ell}$ as $M^{(d)}_{j,\beta_{j},\ell}$, $M^{(\overline{d})}_{j,\beta_{j},\ell}$, and $M^{(\overline{d})}_{j,\nu_{j},\ell}$, which are defined as follows.

[TABLE]

We also assume that there is a start-up delay of $d_{s}$ (in seconds) for the video which is the duration in which the content can be buffered but not played. Table III (in Appendix H) summarizes the key used notation in this paper.

III-C Edge-cache Model

Edge cache $\ell\in\{1,2,\cdots,R\}$, where $R$ is the total number of edge-routers, stores the video content closer to end users. This improves the QoE to end users. We assume a limited cache size at the edge-router (edge-cache) of a maximum capacity of $C_{\ell,e}$ seconds, at edge-router $\ell$. When a file is requested by the user, the edge cache is first checked to see if the file is there completely or partly (in case some other user is watching that content). If the file is not in the edge cache, the space for this video file is created in the edge cache, and a file or some other video files have to be evicted so as not to violate the space constraint.

We consider edge cache policy as follows. The file $i$ is removed from edge cache if it has not been accessed in time $\omega_{i,\ell}$ after its last request time from edge-router $\ell$. The parameter $\omega_{i,\ell}$ is a variable that can be optimized based on the file preference and its placement in the CDN cache. This caching policy is motivated by LRU since the file is evicted if it has not been used in some time in the past. The key advantages of this approach is that (i) It is tunable, in the sense that the parameters $\omega_{i,\ell}$ can be optimized, and (ii) the performance of the policy is easier to optimize as compared to LRU. When a file $i$ is requested, and someone has already requested from edge router $\ell$ in the last $\omega_{i,\ell}$ time units, the file is obtained from the edge router. Even if the file may not be completely in the edge-router yet (not yet finished downloading), the downloaded part is given directly to the new user and the remaining content is delivered as it becomes available to the edge cache. This is akin to multicasting the remaining part of the video to multiple users [32].

An illustration of the evolution of caching policy is illustrated in Figure 3, where the index $\ell$ is omitted since we consider the procedure at a single edge router. Video file $i$ is requested at three times $t_{1}$, $t_{2}$, and $t_{3}$. At $t_{1}$, the file enters the edge cache. Since it had not been requested in $\omega_{i}$ time units, it is evicted. When the file is again requested at $t_{2}$, the space for the file is reserved in the edge cache. The file, when requested at $t_{3}$ is within the $\omega_{i}$ duration from $t_{2}$ and thus will be served from the edge cache. If the file is still not in the edge cache completely, it will obtain the part already there directly while the remaining part will be streamed as it becomes available. Since the file $i$ was not requested for time $\omega_{i}$ after $t_{3}$, the file is again evicted from the edge cache.

We note that the arrival rate of the files is random. Thus, this file eviction policy may not satisfy the maximum edge cache constraint at all times. In order to handle this, we will first assume for the analytical optimization that the probability that the cache capacity of edge-cache $\ell$ is violated is bounded by $\epsilon_{\ell}$ which is small. That could lead us to obtain a rough estimate on the different parameters in the system. The hard constraint on the capacity can be made in run-time, by evicting the files that are closest to be going out based on when they were requested and the corresponding $\omega_{i,\ell}$. This online adaptation will be explained in Appendix J.

III-D Queueing Model and Two-stage probabilistic scheduling

If cache server $j$ is chosen for accessing video file $i$ on edge router $\ell$, the first $L_{j,i}$ chunks are obtained from one of the $e_{j}^{(\ell)}$ parallel streams $PS^{(e,j)}_{\nu_{j},\ell}$. Further, the remaining $L_{i}-L_{j,i}$ chunks are obtained from the data center where a choice of $\beta_{j}$ is made from $1,\cdots,d_{j}$ and the chunks are obtained from the stream $PS^{(d,j)}_{\beta_{j}}$ which after being served from this queue is enqueued in the queue for the stream $PS^{(\overline{d},j)}_{\beta_{j},\ell}$. However, if video file $i$ is already requested at time $t_{i}$ within a window of size $\omega_{i}$, the request will be served from the edge-cache and will not be sent to a higher level in the hierarchy, e.g., cache server.

We assume that the arrival of requests at the edge router $\ell$ for each video $i$ form an independent Poisson process with a known rate $\lambda_{i,\ell}$. In order to serve the request for file $i$, we need to choose three things - (i) Selection of Cache server $j$, (ii) Selection of $\nu_{j}$ to determine one of $PS^{(e,j)}_{\nu_{j},\ell}$ streams to deliver cached content, (iii) Selection of $\beta_{j}$ to determine one of $PS^{(d,j)}_{\beta_{j}}$ streams from the data-center which automatically selects the stream $PS^{(\overline{d},j)}_{\beta_{j},\ell}$ from the cache server, to obtain the non-cached content from the datacenter. Thus, we will use a two-stage probabilistic scheduling to select the cache server and the parallel streams. For a file request at edge-router $\ell$, we choose server $j$ with probability $\pi_{i,j,\ell}$ for file $i$ randomly. Further, having chosen the cache server, one of the $e_{j}$ streams is chosen with probability $p_{i,j,\nu_{j},\ell}$. Similarly, one of the $d_{j}$ streams is chosen with probability $q_{i,j,\beta_{j},\ell}$. We note that these probabilities only have to satisfy

[TABLE]

We note that since file $i$ is removed from the edge cache after time $\omega_{i}$, the requests at the cache server are no longer Poisson. We note that this could be alleviated by assuming that every time the file is requested, the time $\omega_{i}$ is chosen using an exponential distribution. This change of distribution will make the distribution of requests at the cache server Poisson thus alleviating the issue. In the following, we will assume constant $\omega_{i}$, while still approximate the request pattern at cache servers as Poisson which holds when the times for which file remains in the edge cache is chosen using an exponential distribution. This approximation turns out to be quite accurate as will be shown in the evaluation results. Further, such approximations of Poisson arrivals are widely used in the literature in similar fashions. In particular, it is used to characterize the coherence time of an LRU-based caching, e.g., see [33] and references therein. Further, in [34] (Ch.9, page 470) authors approximate the arrivals of new and retransmitted packets in CSMA protocol as Poisson even though they are not due to the dependencies between them.

Since sampling of Poisson process is Poisson, and superposition of independent Poisson processes is also Poisson, we get the aggregate arrival rate at $PS^{(d,j)}_{\beta_{j},\ell}$, $PS^{(\overline{d},j)}_{\beta_{j},\ell}$, and $PS^{(e,j)}_{\nu_{j},\ell}$, denoted as $\Lambda^{(d)}_{j,\beta_{j},\ell}$, $\Lambda^{(\overline{d})}_{j,\beta_{j},\ell}$, and $\Lambda^{(e)}_{j,\nu_{j},\ell}$, respectively are given as follows.

[TABLE]

Lemma 1**.**

If $\omega_{i}$ follows an exponential distribution (i.e., not fixed) with parameter $\nu_{i}$, the probability that the request of video file $i$ is directed to the distributed cache servers and/or central server is given by

[TABLE]

Proof.

Let $\widetilde{t}_{i}$ be the inter-arrival request for video file $i$. Hence, if video file $i$ is requested within $\omega_{i}$ since its last request, video $i$ request will be served from the edge-cache (i.e., $\widetilde{t}_{i}\leq\omega_{i}$). In contrast, if video file $i$ is not at the edge-cache, the request is forwarded to higher hierarchy levels (distributed storage cache and/or central cloud stroage). Since $\widetilde{t}_{i}$ and $\omega_{i}$ are exponentially distributed with paramters $\lambda_{i}$ and $\nu_{i}$, respectively, the probability of directing file $i$ request to the distributed storage cache and/or central cloud stroage is given by $P(\widetilde{t}_{i}>\omega_{i})$. This proves the statement of the Lemma.

∎

Lemma 2**.**

When the service time distribution of datacenter server (first queue) is given by shifted exponential distribution, the arrivals at the cache servers (second queue) are Poisson.

Proof.

The proof is provided in Appendix A. ∎

III-E Distribution of Edge Cache Utilization

We will now investigate the distribution of the edge-cache utilization at any time. This will help us in bounding the probability that the edge cache is more than the capacity of the cache. In the analytic part, we will bound this probability. However, the online adaptations in Appendix J will provide an adaptation to maintain the maximum edge cache capacity constraint at all times.

Let $X_{i,\ell}$ be the random variable corresponding to amount of space in the edge-cache $\ell$ for video file $i$. Since the file arrival rate is Poisson, and the file is in the edge-cache $\ell$ if it has been requested in the last $\omega_{i}$ seconds. Then, $X_{i,\ell}$ is given as

[TABLE]

where $1-e^{-\lambda_{i,\ell}\omega_{i,\ell}}$ is the probability that file $i$ is requested within a window-size of $\omega_{i,\ell}$ time units. The total utilization of the edge-cache $j$ is given as

[TABLE]

The mean and variance of $X_{\ell}$ can be found to be $\sum_{i}^{r}\tau L_{i}(1-e^{-\lambda_{i,\ell}\omega_{i,\ell}}$ and $\sum_{i}^{r}(\tau L_{i})^{2}e^{-\lambda_{i,\ell}\omega_{i,\ell}}(1-e^{-\lambda_{i,\ell}\omega_{i,\ell}})$, respectively. Since $r$ is large, we will approximate the distribution of $X_{j}$ by a Gaussian distribution with mean $\sum_{i}^{r}\tau L_{i}(1-e^{-\lambda_{i,\ell}\omega_{i,\ell}}$ and variance $\sum_{i}^{r}(\tau L_{i})^{2}e^{-\lambda_{i,\ell}\omega_{i,\ell}}(1-e^{-\lambda_{i,\ell}\omega_{i,\ell}})$. This distribution is then used as a constraint in the design of $\omega_{i,\ell}$, where the constraint bounds the probability that the edge cache utilization is higher than the maximum capacity of the edge cache. Since $X_{\ell}$ can be well approximated by a Gaussian distribution, the edge cache utilization, can be probabilistically bounded as follows,

[TABLE]

where $\mu=\sum_{i}^{r}\tau L_{i}(1-e^{-\lambda_{i,\ell}\omega_{i,\ell}})$, $\sigma^{2}=\sum_{i}^{r}(\tau L_{i})^{2}e^{-\lambda_{i,\ell}\omega_{i,\ell}}(1-e^{-\lambda_{i,\ell}\omega_{i,\ell}})$ are the mean and variance, respectively.

IV Stall Duration Tail Probability

This section will characterize the stall duration tail probability using the two-stage probabilistic scheduling and allocation of bandwidth weights. We note that the arrival rates are given in terms of the video files, and the service rate above is provided in terms of segment at each server. The analysis would require detailed consideration of the different segments in a video. In this section, we will assume that the edge-router for the request $\ell$ is known, and thus we omit the subscript/superscript $\ell$ to simplify notations.

In order to find the stall durations, we first consider the case where file $i$ is not in the edge cache and has to be requested from the CDN. We also assume that the cache server $j$ is used, with the streams $\beta_{j}$ and $\nu_{j}$ known. We will later consider the distribution of these choices to compute the overall metric. In order to compute stall durations, we would first calculate the download time of each of the video segment, which accounts for the first $L_{j,i}$ segments at the cache $j$ and the later $L_{i}-L_{j,i}$ segments at the server. After the download times are found, the play times of the different contents are found. The detailed calculations are shown in Appendix B, where the distribution of $T_{i,j,\beta_{j},\nu_{j}}^{\left(g\right)}$, the time that segment $g$ begins to play at the client $i$ given that it is downloaded from $\beta_{j}$ and $\nu_{j}$ queues, is found. The stall duration for the request of file $i$ from $\beta_{j}$ queue, $\nu_{j}$ queue and server $j$, if not in the edge-cache, i.e., $\Gamma^{(i,j,\beta_{j},\nu_{j})}_{U}$ is given as

[TABLE]

as explained in Appendix B. We use this expression to derive a tight bound on the SDTP.

The stall duration tail probability of a video file $i$ is defined as the probability that the stall duration is greater than a pre-defined threshold $\sigma$. Since exact evaluation of stall duration is hard [35, 29], we cannot evaluate $\text{\text{Pr}}\left(\Gamma^{(i)}_{tot}\geq\sigma\right)$ in closed-form, where $\Gamma^{(i)}_{tot}$ is random variable indicating the overall stall duration for file $i$. In this section, we derive a tight upper bound on the SDTP through the two-stage Probabilistic Scheduling as follows.

We first note that the expression in equation (76) (Appendix B) accounts only for the stalls that would be incurred if the video segments are not accessed from the edge-cache (including stored, or multicasted). However, the user would experience lower stalls if the requested content is accessed from the edge-cache. Thus, we need an expectation over the choice of whether the file is accessed from the edge server, and the choice of $(j,\beta_{j},\nu_{j})$ in addition to the queue statistics. For a video file $i$ requested at time $\widetilde{t}_{i}$ after the last request for file $i$, the stall duration for the request of file $i$ can be expressed as follows:

[TABLE]

where $\overset{d}{=}$ means equal in distribution. This is because if file $i$ is requested again within $\omega_{i}$ time, then the multicast or stored file can lead to the reduced stall duration based on how much time has passed since the last request. Further, if the file has not been requested in the last $\omega_{i}$ time units, then the file has to be obtained from the CDN, and thus the expression of random variable $\Gamma_{tot}^{(i)}$ also includes randomness over the choice of $(j,\beta_{j},\nu_{j})$ in this case. From (18), we can obtain the following result.

Lemma 3**.**

For a given choice of $(j,\beta_{j},\nu_{j})$, $\mathbb{E}\left[e^{h_{i}\Gamma_{tot}^{(i)}}\right]$ is bounded as

[TABLE]

where $c=1-e^{-\lambda_{i}\omega_{i}}$, $a=e^{-\lambda_{i}\omega_{i}}$, b=\Biggl{[}1-\frac{\lambda_{i}}{\lambda_{i}+h_{i}}\left(1-e^{-\left(\lambda_{i}+h_{i}\right)\omega_{i}}\right)\Biggr{]}, $\widetilde{c}=c/b$ and $\widetilde{a}=a/b$.

Proof.

The proof is provided in Appendix C. ∎

We next derive $\mathbb{E}\left[e^{h_{i}D_{i,j,\beta_{j},\nu_{j}}^{(v)}}|(j,\beta_{j},\nu_{j})\right]$ using the following two lemmas, which will be used in the main result. The key idea is that we characterize the download and play times of each segments and use them in determining the SDTP of each video file request.

Lemma 4**.**

For $v\leq L_{j,i}$, $\mathbb{E}\left[e^{h_{i}D_{i,j,\beta_{j},\nu_{j}}^{(v)}}|(j,\beta_{j},\nu_{j})\right]$ is given by

[TABLE]

Proof.

The proof follows from (53) in Appendix B by replacing $g$ by $v$ and rearranging the terms in the result. ∎

Lemma 5**.**

For $v\geq(L_{j,i}+1)$, $\mathbb{E}\left[e^{h_{i}D_{r}^{(v)}}|(j,\beta_{j},\nu_{j})\right]$ is given by

[TABLE]

where ${\mathbb{E}}\left[e^{h_{i}U_{i,j,\beta_{j},v,L_{j,i}}}|(j,\beta_{j},\nu_{j})\right]$ and $B_{j,\beta_{j}}^{(d)}$ are given in Appendix B, equations (65), and (67), respectively.

Proof.

The proof is provided in Appendix D. ∎

Corollary 1**.**

The (expected) time to the first chunk (TTFC) can be obtained from equation (101) (Lemma 3) by setting $d_{s}=0$ and $g_{i}=1$.

Using these expressions, the following theorem summarizes the stall duration tail probability for file $i$. We include the edge router index $\ell$ in all the expressions in the result for the ease of using it in the following section.

Theorem 1**.**

The stall distribution tail probability for video file $i$ requested through edge router $\ell$ is bounded by

[TABLE]

for $\rho_{j,\beta_{j}}^{(d)}<1$, $\rho_{j,\beta_{j},\ell}^{(\overline{d})}<1$, $\rho_{j,\nu_{j},\ell}^{(e)}<1$, where the auxiliary variables in the statement of the Theorem are defined as

[TABLE]

[TABLE]

[TABLE]

Proof.

The detailed steps are provided in Appendix E.

∎

We note that $\delta^{(e,\ell)}=\delta^{(\overline{d},\ell)}=0$, if the storage server nodes are not hosting the requested video files and $\delta^{(\overline{d},d,\ell)}$ has nonzero value only if some sWe can also derive the mean stall duration for video file $i$ in a similar fashion. The interested reader is referred to Appendix I for detailed treatment of this metric.

V Optimization Problem Formulation and Proposed Algorithm

V-A Problem Formulation

We define $\boldsymbol{\pi}=(\pi_{i,j,\ell}\forall i=1,\cdots,r\text{ and }j=1,\cdots,m,\ell=1,\cdots,R),\boldsymbol{p}=(p_{i,,j,\nu_{j},\ell},(\nu_{j},\ell)\in\{(\nu_{j},\ell):\nu_{j}\in\{1,\ldots,e_{j}^{(\ell)}\},\ell\in\{1,\dots,R\}\},\forall i=1,\cdots,r\,,j=1,\cdots,m\,,\,\nu_{j}=1,\ldots e_{j}^{(\ell)},\ell=1,\cdots,R),\boldsymbol{q}=(q_{i,,j,\beta_{j},\ell}\forall i=1,\cdots,r\,,j=1,\cdots,m\,,\,\beta_{j}=1,\ldots d_{j},\ell=1,\cdots,R),\boldsymbol{h}=\left(h_{1},t_{2},\ldots,h_{r}\right)\,,\boldsymbol{w}^{(\overline{d})}=(w_{j,1,\ell}^{(\overline{d})},\ldots,w_{j,d_{j},\ell}^{(d)},\text{ and }j=1,\cdots,m,\ell=1,\cdots,R)\,,\boldsymbol{w}^{(e)}=(w_{j,1,\ell}^{(e)},w_{j,2,\ell}^{(e)},\ldots,w_{j,e_{j}^{(\ell)},\ell}^{(e)},\text{ and }j=1,\cdots,m,\ell=1,\ldots,R),\,\boldsymbol{w}^{(d)}=(w_{j,1,\ell}^{(d)},\ldots,w_{j,d_{j},\ell}^{(d)},\text{ and }j=1,\cdots,m),\boldsymbol{L}=(L_{j,i},\,\forall i=1,\,\text{ and }j=1,\cdots,m)$ and $\boldsymbol{\omega}=(\omega_{i,\ell},\forall i,\ell)$. Our goal is to minimize the SDTP over the choice of cache and datacenter access decisions, bandwidth allocation weights, portion (number) of cached segments, time window over which we maintain the video files at edge-cache and auxiliary bound parameters.

To incorporate for weighted fairness and differentiated services, we assign a positive weight $\kappa_{i,\ell}$ for each file $i$. Without loss of generality, each file $i$ is weighted by the arrival rate $\lambda_{i,\ell}$ in the objective (so larger arrival rates are weighted higher). However, any other weights can be incorporated to accommodate for weighted fairness or differentiated services. Let $\overline{\lambda}=\sum_{i,\ell}\lambda_{i,\ell}$ be the total arrival rate. Hence, $\kappa_{i,\ell}=\lambda_{i,\ell}/\overline{\lambda}$ is the ratio of file $i$ requests. Hence, the objective is the minimization of stall duration tail probability, averaged over all the file requests, and is given as $\sum_{i,\ell}\frac{\lambda_{i,\ell}}{\overline{\lambda}}\,\text{\text{Pr}}\left(\Gamma^{(i)}\geq\sigma\right)$. By using the expression for SDTP in Section IV, the optimization problem can be formulated as follows.

[TABLE]

[TABLE]

Here, in (36), equations $(1)-(3)$ give the feasibility constraints on the bandwidth allocation, while equations $(4)-(6)$ define the MGFs of the service time distributions, equations $(7)-(10)$ give the feasibility of the two-stage probabilistic scheduling and $(11)-(12)$ define the arrival rates at the different queues. Constraints (37)–(39) ensure the stability of the systems queue (do not blow up to infinity). Constraints (41)–(44) ensure that the moment generating functions exist. We note that some optimization variables can be combined to form a single optimization variable which results in having only five independent and separable variables as shown below. In the next subsection, we will describe the proposed algorithm for this optimization problem.

V-B Proposed Algorithm

We first note that the two-stage probabilistic scheduling variables are independent and separable, thus we can combine them and define a single variable $\widetilde{\boldsymbol{\pi}}$ such that $\widetilde{\boldsymbol{\pi}}=\left(\boldsymbol{\pi}\,,\boldsymbol{\,p}\,\,,\boldsymbol{q}\right)$. Similarly, since the bandwidth allocation weights are independent and separable, we concatenate them in a single optimization variable $\boldsymbol{w}$, where $\boldsymbol{w}=\left(\boldsymbol{w}^{(e)}\,,\boldsymbol{\,w}^{(\overline{d})}\,\,,\boldsymbol{w}^{(d)}\right)$. Hence, the weighted SDTP optimization problem given in (35)-(45) is optimized over five set of variables: server and PSs scheduling probabilities $\widetilde{\boldsymbol{\pi}}$ (two-stage scheduling probabilities), auxiliary parameters $\boldsymbol{h}$, bandwidth allocation weights $\boldsymbol{w}$, cache placement $\boldsymbol{L}$, and edge cache window size optimization $\boldsymbol{\omega}$.

Clearly, the problem is non-convex in all the parameters jointly, which can be easily seen in the terms which are product of the different variables. Since the problem is non-convex, we propose an iterative algorithm to solve the problem. The proposed algorithm divides the problem into five sub-problems that optimize one variable while fixing the remaining four. These sub-problems are labeled as (i) Server and PSs Access Optimization: optimizes $\widetilde{\boldsymbol{\pi}}$, for given $\boldsymbol{h}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$, (ii) Auxiliary Variables Optimization: optimizes $\boldsymbol{h}$ for given $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$, (iii) Bandwidth Allocation Optimization: optimizes $\boldsymbol{w}$ for given $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$. (iv) Cache Placement Optimization: optimizes $\boldsymbol{L}$ for given $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{\omega}$, and $\boldsymbol{w}$, (v) Edge-cache Window Size Optimization: optimizes $\boldsymbol{\omega}$ for given $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{L}$, and $\boldsymbol{w}$. The algorithm is summarized as follows.

Initialization: Initialize $\boldsymbol{h}$, $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$ and $\boldsymbol{L}$ in the feasible set. 2. 2.

While Objective Converges

1. (a)

Run Server Access Optimization using current values of $\boldsymbol{h}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$ to get new values of $\widetilde{\boldsymbol{\pi}}$ 2. (b)

Run Auxiliary Variables Optimization using current values of $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$ to get new values of $\boldsymbol{h}$ 3. (c)

Run Bandwidth Allocation Optimization using current values of $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{L}$, and $\boldsymbol{\omega}$, to get new values of $\boldsymbol{w}$. 4. (d)

Run Cache Placement Optimization using current values of $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{w}$, and $\boldsymbol{\omega}$ to get new values of $\boldsymbol{L}$. 5. (e)

Run Edge-cache Window Size Optimization using current values of $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{w}$, and $\boldsymbol{L}$ to get new values of $\boldsymbol{\omega}$.

The proposed algorithm performs an alternating optimization over the different aforementioned dimensions, such that each sub-problem is shown to have convex constraints and thus can be efficiently solved using the iNner cOnVex Approximation (NOVA) algorithm proposed in [15]. The subproblems are explained in detailed in Appendix F.

We first initialize $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{w}$, $\boldsymbol{h}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$ $\forall i,j,\nu_{j},\beta_{j}$ such that the choice is feasible for the problem. Then, we do alternating minimization over the five sub-problems defined above. Since each sub-problem can only decrease the objective (properties of convergence of subproblems to a stationary point is given in Appendix F) and the overall problem is bounded from below, we have the following result.

Theorem 2**.**

The proposed algorithm converges to a stationary solution.

Appendix J describes how our algorithm can be used in an online fashion to keep track of the systems dynamics at the edge-cache.

VI Implementation and Evaluation

In this section, we evaluate our proposed algorithm for weighted stall duration tail probability.

VI-A Testbed Configuration and Parameter Setup

We construct an experimental environment in a virtualized cloud environment managed by Openstack [16] to investigate our proposed SDTP framework. We allocated one VM for an origin server and 5 VMs for cache servers intended to simulate two locations (e.g., different states). We implement the proposed online caching mechanism in the edge cache that takes the inputs of $\omega_{i,\ell}$ at each edge router. When a video file is requested, it is stored in the edge-cache for a window size of $\omega_{i,\ell}$ time units (unless requested again in this window). For the future requests within $\omega_{i}$ or concurrent user requests, the requests for the video chunks are served from the edge-cache, and thus future/concurrent users would experience lower stall duration. If the file can be accessed from the edge router, higher caching level is not used for this request which consequently reduces the traffic at the core backbone servers. If the file cannot be accessed from the edge router, it goes to the distributed cache. We assume some segments, i.e., $L_{j,i}$, of video file $i$ are stored in the distributed cache node $j$, and are served from the cache nodes. The non-cached segments are served from the data-center. The schematic of our testbed is illustrated in Figure 4. Since the two edge-routers are likely in different states, they may not share the cache servers which is the setup we study in the experiments. We note that the theoretical approach proposed earlier is general and can work with shared cache servers across multiple edge routers.

One VM per location is used for generating client workloads. Table II summarizes a detailed configuration used for the experiments. For client workload, we exploit a popular HTTP-traffic generator, Apache JMeter, with a plug-in that can generate traffic using HTTP Streaming protocol. We assume the amount of available bandwidth between origin server and each cache server is 200 Mbps, 500 Mbps between cache server 1/2 and edge router 1, and 300 Mbps between cache server 3/4/5 and edge router 2. In this experiment, to allocate bandwidth to the clients, we throttle the client (i.e., JMeter) traffic according to the plan generated by our algorithm. We consider $1000$ threads (i.e., users) and set $e_{j}^{(\ell)}=40$ for all $\ell=1,2$, $d_{j}=20$. Segment size $\tau$ is set to be equal to $8$ seconds. Each edge cache is assumed to have a capacity, equivalent to $15\%$ of the total size of the video files. Further, distributed cache servers can store up to $35\%$ out of the total number of video file segments. The values of $\alpha_{j}$ and $\eta_{j}$ are summarized in Table I.

Video files are generated based on Pareto distribution [40] (as it is a commonly used distribution for file sizes [41]) with shape factor of $2$ and scale of $300$, respectively. While we stick in the experiment to these parameters, our analysis and results remain applicable for any setting given that the system maintains stable conditions under the chosen parameters. Since we assume that the video file sizes are not heavy-tailed, the first $500$ file-sizes that are less than 60 minutes are chosen. When generating video files, the size of each video file is rounded up to the multiple of $8$ seconds. For the arrival rates, we use the data from our production system for 500 hot files from two edge routers, and use those arrival rates. The aggregate arrival rates at edge router 1 and router 2 are $\Lambda_{1}=0.01455s^{-1}$, $\Lambda_{2}=0.02155s^{-1}$, respectively.

In order to generate the policy for the implementation, we assume uniform scheduling, $\pi_{i,j}=k/n$, $p_{j,\nu_{j}}=1/e_{j}$, $q_{j,\beta_{j}}=1/d_{j}$. Further, we choose $t_{i}=0.01$, $w_{j,\nu_{j}}^{(e)}=1/e_{j}$, $w_{j,\beta_{j}}^{(\overline{d})}=1/d_{j}$ and $w_{j,\beta_{j}}^{(d)}=1/d_{j}$. However, these choices of the initial parameters may not be feasible. Thus, we modify the parameter initialization to be closest norm feasible solutions. Using the initialization, the proposed algorithm is used to obtain the parameters. These parameters are then used to control the bandwidth allocation, distributed cache content placement, the probabilistic scheduling parameters, and the edge caching window sizes. Based on these parameters, the proposed online algorithm is implemented. Since we assume the arrivals of video files are Poisson (and hence inter-arrival time is exponential with $\lambda_{i}$ for file $i$), we generate a sequence of $10000$ video file arrivals/requests corresponding to the different files at each edge router. Upon an arrival of a video file at edge-cache, we apply our proposed online mechanism. For each segment, we used JMeter built-in reports to estimate the downloaded time of each segment and then plug these times into our model to obtain the stall duration which will be used for evaluation of the proposed method.

VI-B Baselines

We compare our proposed approach with multiple strategies, which are described as follows.

Projected Equal Server-PSs Scheduling, Optimized Auxiliary variables, Cache Placement, Edge-cache Window-Size, and Bandwidth Wights (PEA): Starting with the initial solution mentioned above, the problem in (35) is optimized over the choice of $\boldsymbol{h}$, $\boldsymbol{w}$, $\boldsymbol{L}$, and $\boldsymbol{\omega}$ (using Algorithms 2, 3, 4, and 5 respectively) using alternating minimization. Thus, the values of $\pi_{i,j}$, $p_{i,j,\nu_{j}}$, and $q_{i,j,\beta_{j}}$ will be approximately close to $k/n$, $1/e_{j}$, and $1/d_{j}$, respectively, for all $i,j,\nu_{j},\beta_{j}$. 2. 2.

Projected Proportional Service-Rate, Optimized Auxiliary variables, Bandwidth Wights, Edge-cache Window-Size, and Cache Placement (PSP): In the initialization, the access probabilities among the servers, are given as $\pi_{i,j}=\frac{\mu_{j}}{\sum_{j}\mu_{j}},\,\forall i,j$. This policy assigns servers proportional to their service rates. The choice of all parameters are then modified to the closest norm feasible solution. Using this initialization, the problem in (35) is optimized over the choice of $\boldsymbol{h}$, $\boldsymbol{w}$, $\boldsymbol{L}$, and $\boldsymbol{\omega}$, (using Algorithms 2, 3, 4, and 5, respectively) using alternating minimization. 3. 3.

Projected Equal Caching, Optimized Scheduling Probabilities, Auxiliary variables and Bandwidth Allocation Weights (PEC): In this strategy, we divide the cache size equally among the video files. Thus, the size of each file in the cache is the same (unless file is smaller than the cache size divided by the number of files). Using this initialization, the problem in (35) is optimized over the choice of $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{w}$, and $\boldsymbol{\omega}$ (using Algorithms 1, 2, 3, and 5, respectively) using alternating minimization. 4. 4.

Caching Hot Files, Optimized Scheduling Probabilities, Auxiliary variables, Edge-cache Window-Size, and Bandwidth Allocation Weights (CHF): In this strategy, we cache entirely the files that have the largest arrival rates in the storage cache server. Such hot file caching policies have been studied in the literature, see [12] and references therein. Using this initialization, the problem in (35) is optimized over the choice of $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{w}$, and $\boldsymbol{\omega}$ (using Algorithms 1, 2, 3, and 5, respectively) using alternating minimization. 5. 5.

Caching based on Least-Recently-Used bases at edge-cache and Caching-Hottest files at storage nodes, Optimized Scheduling Probabilities, Auxiliary variables, Storage Cache Placement, and Bandwidth Allocation Weights (LRU): In this strategy, a file is entirely cached in the edge-cache servers upon request if space permits; otherwise, the least-recently used file(s) is removed first to evacuate the needed space for the new file. Further, the hottest files are partially cached in the distributed storage cache servers. Such hot file caching policies have been studied in the literature, e.g., [12] and references therein. Using this initialization, the problem in (35) is optimized over the choice of $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, and $\boldsymbol{w}$, (using Algorithms 1, 2, and 3, respectively) using alternating minimization. 6. 6.

Caching at edge-cache based on adaptSize policy [42] and Caching-Hottest files at storage nodes, Optimized Scheduling Probabilities, Auxiliary variables, Storage Cache Placement, and Bandwidth Allocation Weights (adaptSize): This policy is a probabilistic admission policy in which a video file is admitted into the cache with probability $e^{-size/c}$ so as larger objects are admitted with lower probability and the parameter $c$ is tuned to maximize the object hit rate (OHR), defined as the probability that a requested file is found in the cache. In particular, given a $c$ and an estimate on the arrival rate for the requests for each video file, one can estimate the probability that a given file will be served from the edge-cache. One can then use these probabilities to compute the OHR as a function of $c$ and then optimize. The value of $c$ is recomputed after a certain number of file requests, using a sliding window approach. We refer the reader to [42] for a more in-depth description. 7. 7.

Caching at edge-cache based on variant of LRU policy [33], Caching-Hottest files at storage nodes, Optimized Scheduling Probabilities, Auxiliary variables, Storage Cache Placement, and Bandwidth Allocation Weights ($xLRU$): We denote by $xLRU$ one of the these policies: $qLRU$, $kLRU$, and $k$Random. A $qLRU$ policy is the same as LRU except that files are only added with probability $q$. In $kLRU$, requested files must traverse $k-1$ additional virtual LRU caches before it is added to the actual cache. kRandom is the same as $kLRU$ except files are evicted from the cache at random. The other optimization parameters are optimized the same way as in the adaptSize policy.

VI-C Experimental Results

SDTP performance for different $\sigma$: Figure 5 shows the decay of weighted SDTP $\sum_{i=1}^{r}\frac{\lambda_{i}}{\overline{\lambda}_{i}}\mathbb{P}(\Gamma^{(i)}>\sigma)$ with $\sigma$ (in seconds) for the considered policies. Notice that SDTP Policy solves the optimal weighted stall tail probability via proposed alternating optimization algorithm. Also, this figure represents the complementary cumulative distribution function (ccdf) of the proposed algorithm as well as the selected baselines. We further observe that uniformly accessing servers and simple service-rate-based scheduling are unable to optimize the request scheduler based on factors like chunk placement, request arrival rates, different stall weights, thus leading to much higher SDTP. Moreover, the figure shows that an entire video file does not have to be present in the edge-cache. That’s because when the user requests a cached video, it is served by first sending the portion of the video locally present at edge-cache while obtaining the remainder from the distributed cache servers and/or the origin server, and transparently passing it on to the client. In addition, we see that the analytical (offline) SDTP is very close to the actual (online) SDTP measurement on our testbed. Further, since adaptSize policy does not intelligently incorporate the arrival rates in adding/evicting the video files, it fails to significantly reduce the SDTP. To the best of our knowledge, this is the first work to jointly consider all key design degrees of freedom, including bandwidth allocation among different parallel streams, cache content placement, the request scheduling, window-size of the edge-cache and the modeling variables associated with the SDTP bound.

Arrival Rates Comparisons: Figure 6 shows the effect of increasing system workload, obtained by varying the arrival rates of the video files from $0.01s^{-1}$ to $0.03s^{-1}$ with an increment step of $0.002s^{-1}$ on the SDTP. We notice a significant improvement of the QoE metric with the proposed strategy as compared to the baselines. Further, the gap between the analytical offline bound and actual online SDTP is small which validates the tightness of our proposed SDTP bound. Further, while our algorithm optimizes the system parameters offline, this figure shows that an online version of our algorithm can be used to keep track of the systems dynamics and thus achieve an improved performance.

Effect of Number of files: Figure 7 shows the impact of varying the number of files from $150$ to $550$ on the weighted SDTP for the online algorithm. Clearly, weighted SDTP increases with the number of files, which brings in more workload (i.e., higher arrival rates). However, our optimization algorithm optimizes new files along with existing ones to keep overall weighted SDTP at a low level. We note that the proposed optimization strategy effectively reduces the tail probability and outperforms the considered baseline strategies. Thus, joint optimization over all optimization parameters help reduce the tail probability significantly. Also, the gap between online and offline performance is almost negligible which reflects the robustness of our algorithm.

Additional performance evaluation is provided in Appendix K and Appendix L.

VII Conclusion

This paper proposes a CDN-based edge-cache-aided over-the-top multicast video streaming system, where the video content is partially stored on distributed cache servers and access-dependent online edge caching strategy is used at the edge-cache. Further, this paper optimizes the weighted stall duration tail probability by considering two-stage probabilistic scheduling for the choice of servers and the parallel streams between the server and the edge router. Using the two-stage probabilistic scheduling and the edge caching mechanism, upper bound on the stall duration tail probability is characterized. Further, an optimization problem that minimizes the weighted stall duration tail probability is formulated, over the choice of two-stage probabilistic scheduling, bandwidth allocation, cache placement, edge-cache parameters, and the auxiliary variables in the bound. An efficient algorithm is proposed to solve the optimization problem and the experimental results on a virtualized cloud system managed by Openstack depict the improved performance of our proposed algorithm as compared to the considered baselines. Possible extensions to accommodate multiple quality levels and different chunk sizes are discussed in Appendix M. However, a complete treatment of adaptive bit-rate video streaming is left as a future work.

VIII Acknowledgment

We would like to thank Prof. Tian Lan from GWU for helpful discussions related to this work.

Appendix A Proof of Lemma 2

We note that the arrivals at the second queue in two $M/M/1$ tandem networks are Poisson is well known in the queueing theory literature [43]. The service distribution of the first queue in this paper is a shifted exponential distribution. We note that the deterministic shift also does not change the distribution since the number of arrivals in any time window in the steady state will be the same. Thus, the arrival distribution in the second queue will still be Poisson.

Appendix B Download and Play Times of a Segment not Requested in $\omega_{i}$

Since we assume the edge-router index, we will omit $\ell$ in this section. In order to characterize the stall duration tail probability, we need to find the download time and the play time of different video segments, for any server $j$ and streams with the choice of $\beta_{j}$ and $\nu_{j}$, assuming that they are not requested in $\omega_{i}$. The optimization over these decision variables will be considered in this paper.

B-A Download Times of first $L_{j,i}$ Segments

We consider a queueing model, where $W_{j,\nu_{j}}^{(e)}$ denotes the random waiting time of all video files in the queue of $PS^{(e,j)}_{\nu_{j}}$ before file $i$ request is served, and $Y_{i,j,\nu_{j}}^{(e,g)}$ be the (random service time of a coded chunk $g$ for file $i$ from server $j$ and queue $\nu_{j}$. Then, for $g\leq L_{j,i}$, the random download time of the first $L_{j,i}$ segments $g\in\{1,\ldots,L_{j,i}\}$ if file $i$ from stream $PS^{(e,j)}_{\nu_{j}}$ is given as

[TABLE]

Since video file $i$ consists of $L_{j,i}$ segments stored at cache server $j$, the total service time for video file $i$ request at queue $PS^{(e,j)}_{\nu_{j}}$, denoted by $ST_{i,j,\nu_{j}}$, is given as

[TABLE]

Hence, the service time of the video files at the parallel stream $PS^{(e,j)}_{\nu_{j}}$ is given as

[TABLE]

We can show that the moment generating function of the service time for all video files from parallel stream $PS^{(e,j)}_{\nu_{j}}$ is given by

[TABLE]

Further, based on our 2-stage scheduling policy, the load intensity at $PS^{(e,j)}_{\nu_{j}}$ is as follows

[TABLE]

Since the arrival is Poisson and the service time is shifted-exponentially distributed, the moment generating function (MGF) of the waiting time at queue $PS^{(e,j)}_{\nu_{j}}$ can be calculated usingthe Pollaczek-Khinchine formula, i.e.,

[TABLE]

From the MGF of $W_{j,\nu_{j}}^{(e)}$ and the service time, the MGF of the download time of segment $g$ from the queue $PS^{(e,j)}_{\nu_{j}}$ for file $i$ is then

[TABLE]

We note that the above is defined only when MGFs exist, i.e.,

[TABLE]

B-B *Download Times of last $(L_{i}-L_{j,i})$ Segments *

Since the later video segments $(L_{i}-L_{j,i})$ are downloaded from the data center, we need to schedule them to the $\beta_{j}$ streams using the proposed probabilistic scheduling policy. We first determine the time it takes for chunk $g$ to depart the first queue (i.e., $\beta_{j}$ queue at datacenter). For that, we define the time of chunk $g$ to depart the first queue as

[TABLE]

where $W_{j,\beta_{j}}^{(d)}$ is the waiting time from $PS^{(d,j)}_{\beta_{j}}$ for the earlier video segments, and $Y^{(d,v)}_{j,\beta_{j}}$ is the service time for obtaining segment $v$ from the queue of $PS^{(d,j)}_{\beta_{j}}$. Using similar analysis for that of deriving the MGF of download time of chunk $g$ as in the last section, we obtain

[TABLE]

where the load intensity at queue $\beta_{j}$ at datacenter, $\rho_{j,\beta_{j}}^{(d)}$

[TABLE]

To find the download time of video segments from the second queue (at cache server $j$), we notice that the download time for segment $g$ includes the waiting to download all previous segments and the idle time if the segment $g$ is not yet downloaded from the first queue ($PS^{(d,j)}_{\beta_{j}}$), as well as the service time of the segment from $PS^{(\overline{d},j)}_{\beta_{j}}$. Then, the download time of the video segments from the second queue (i.e.,$PS^{(\overline{d},j)}_{\beta_{j}}$) can be derived by a set of recursive equations, with download time of the first (initial) segment $(L_{j,i}+1)$ as

[TABLE]

where $W_{j,\beta_{j}}^{(\overline{d})}$ is the waiting time from queue $PS^{(\overline{d},j)}_{\beta_{j}}$ for the previous video segments, and $Y^{(\overline{d},v)}_{j,\beta_{j}}$ is the required service time for obtaining segment $v$ from the queue of $PS^{(\overline{d},j)}_{\beta_{j}}$. The download time of the following segments ($g>L_{j,i}+1$) is given by the following recursive equation

[TABLE]

With the above recursive equations from $y=L_{j,i}$ to $y=g$, we can obtain that

[TABLE]

where

[TABLE]

Similarly, for $y>L_{j,i}$, we have

[TABLE]

It is easy to see that the moment generating function of $U_{i,j,\beta_{j},g,y}$ for $y=L_{j,i}$ is given by

[TABLE]

where the load intensity at queue $\beta_{j}$ at cache server $j$, $\rho_{j,\beta_{j}}^{(\overline{d})}$ is given by

[TABLE]

Similarly, the moment generating function of $U_{i,j,\beta_{j},g,y}$ for $y>L_{j,i}$ is given as

[TABLE]

where $\overline{W}_{j,\beta_{j}}^{(d)}$ $\overline{W}_{j,\beta_{j}}^{(d)}=\frac{(1-\rho_{j,\beta_{j}}^{(d)})tB_{j,\beta_{j}}^{(d)}(t)}{t-\Lambda_{j,\beta_{j}}^{(d)}(B_{j,\beta_{j}}^{(d)}(t)-1)}$. We further note that these moment generating functions are only defined when the MGF functions exist, i.e.,

[TABLE]

B-C Play Times of different Segments

Next, we find the play time of different video segments. Recall that $D_{i,j,\beta_{j},\nu_{j}}^{(g)}$ is the download time of segment $g$ from $\nu_{j}$ and $\beta_{j}$ queues at client $i$. We further define $T_{i,j,\beta_{j},\nu_{j}}^{\left(g\right)}$ as the time that segment $g$ begins to play at the client $i$, given that it is downloaded from $\beta_{j}$ and $\nu_{j}$ queues. This start-up delay of the video is denoted by $d_{s}$. Then, the first segment is ready for play at the maximum of the startup delay and the time that the first segment can be downloaded. This means

[TABLE]

For $1<g\leq L_{i}$, the play time of segment $g$ of video file $i$ is given by the maximum of (i) the time to download the segment and (ii) the time to play all previous segment plus the time to play segment $g$ (i.e., $\tau$ seconds). Thus, the play time of segment $g$ of video file $i$, when requested from server $j$ and from $\nu_{j}$ and $\beta_{j}$ queues, can be expressed as

[TABLE]

This results in a set of recursive equations, which further yield by

[TABLE]

where $\mathcal{F}_{i,j,\beta_{j},\nu_{j},z}$ is expressed as

[TABLE]

We now get the MGFs of the $\mathcal{F}_{i,j,\nu_{j},\beta_{j},z}$ to use in characterizing the play time of the different segments. Towards this goal, we plug Equation (74) into $\mathbb{E}\left[e^{t\mathcal{F}_{i,j,\nu_{j},\beta_{j},z}}\right]$ and obtain

[TABLE]

where $\mathbb{E}\left[e^{tD_{i,j,\beta_{j},\nu_{j}}^{(z-1)}}\right]$ can be calculated using equation (53) when $1<z\leq(L_{j,i}+1)$ and using equation (68) when $z>L_{j,i}+1$ .

The last segment should be completed by time $d_{s}+L_{i}\tau$ (which is the time at which the playing of the $L_{i}-1$ segment finishes). Thus, the difference between the play time of the last segment $T_{i,j,\beta_{j},\nu_{j}}^{(L_{i})}$ and $d_{s}+\left(L_{i}-1\right)\tau$ gives the stall duration. We note that the stalls may occur before any segment and hence this difference will give the sum of durations of all the stall periods before any segment. Thus, the stall duration for the request of file $i$ from $\beta_{j}$ queue, $\nu_{j}$ queue and server $j$, i.e., $\Gamma^{(i,j,\beta_{j},\nu_{j})}_{U}$ is given as

[TABLE]

Next, we use this expression to derive a tight bound on the SDTP.

Appendix C Proof of Lemma 3

From (18), we can get

[TABLE]

By taking the expectation of both sides in (77), we can write

[TABLE]

where the expectation in the second case is over the choice of $(j,\beta_{j},\nu_{j})$ in addition to the queue statistics with arrival and departure rates. Since the arrivals at edge-cache of video files are Poisson, the time till first request for file $i$, i.e., $\widetilde{t}_{i}$, is exponentially distributed with rate $\lambda_{i}$. By averaging over $\widetilde{t}_{i}$, we have

[TABLE]

Performing the integration and simplifying the expressions, we get

[TABLE]

This can be further simplified as follows.

[TABLE]

where $(a)$ and $(b)$ follow by setting $c=1-e^{-\lambda_{i}\omega_{i}}$, $a=e^{-\lambda_{i}\omega_{i}}$, b=\Biggl{[}1-\frac{\lambda_{i}}{\lambda_{i}+h_{i}}\left(1-e^{-\left(\lambda_{i}+h_{i}\right)\omega_{i}}\right)\Biggr{]}, $\widetilde{c}=c/b$ and $\widetilde{a}=a/b$. We also recall that the expectation in ${\mathbb{E}}\left[e^{h_{i}\Gamma_{U}^{(i,j,\beta_{j},\nu_{j})}}\right]$ is over the choice of $(j,\beta_{j},\nu_{j})$ and the queue arrival/departure statistics.

Appendix D Proof of Lemma 5

We have

[TABLE]

where $(a)$ follows from (62), the inequality above follows by replacing the $\text{max}_{y}(.)$ by $\sum_{y}(.)$. Moreover, the last step follows from (68). Hence, we can write

[TABLE]

this proves the statement of the Lemma.

Appendix E Proof of Theorem 1

The SDTP for the request of file $i$ can be bounded using Markov Lemma as follows

[TABLE]

This can be further simplified as follows

[TABLE]

where $\mathcal{F}_{i,j,\beta_{j},\nu_{j},z}$ and $D_{i,j,\beta_{j},\nu_{j}}^{(v)}$ are given in Appendix B in Equations (74) and (53), respectively. Further, $(c)$ follows from (19), $(d)$ follows from (73) and by setting $\overline{c}=\widetilde{c}\,e^{-h_{i}\sigma}$, $\overline{a}=\widetilde{a}\,e^{-h_{i}(\sigma+d_{s}+(L_{i}-1)\tau)}$, $(e)$ follows by upper bounding the maximum by the sum, and (f) follows from (74). Using the two-stage probabilistic scheduling, the SDTP for video file $i$ is further bounded by

[TABLE]

Using Lemmas 4 and 5 for $\mathbb{E}\left[e^{h_{i}D_{r}^{(v)}}|(j,\beta_{j},\nu_{j})\right]$, we obtain the following.

[TABLE]

[TABLE]

where step $(f)$ follows by substitution of the moment generating functions, and the remaining of the steps use the sum of geometric and Arithmetico-geometric sequences. Note that the subscript $\ell$ is omitted in the above derivation for simplicity. Further, $\delta^{(e)}=\frac{\widetilde{M}_{j,\nu_{j}}^{(e)}(h_{i})(1-\rho_{j,\beta_{j}}^{(e)})t_{i}((\widetilde{M}_{j,\nu_{j}}^{(e)}(h_{i}))^{L_{j,i}}-1)}{(h_{i}-\Lambda_{j,\beta_{j}}^{(e)}(B_{j,\beta_{j}}^{(e)}(h_{i})-1))(\widetilde{M}_{j,\nu_{j}}^{(e)}(h_{i}))-1)}$, $\delta^{(\overline{d})}=\frac{(1-\rho_{j,\beta_{j}}^{(\overline{d})})t_{i}(\widetilde{M}_{j,\nu_{j}}^{(\overline{d})}(h_{i}))^{L_{j,i}-L_{i}}}{h_{i}-\Lambda_{j,\beta_{j}}^{(\overline{d})}(B_{j,\beta_{j}}^{(\overline{d})}(h_{i})-1)}$, $\!\delta^{(d,\overline{d})}\!=\!\!\!\!\!\gamma^{(d)}\!\!\left(\!\!\!\frac{(\widetilde{M}_{j,\beta_{j}}^{(d,\overline{d})}(h_{i}))^{L_{i}-L_{j,i}}\!\!-(L_{i}\!\!-L_{j,i}\!\!)}{(\widetilde{M}_{j,\beta_{j}}^{(d,\overline{d})}(h_{i}))-1}\!\!+\!\xi_{i,j,\beta_{j}}^{(d,\overline{d})}\!\!\right)$, $\!\xi_{i,j,\beta_{j}}^{(d,\overline{d})}=\!\!\!\!\!\frac{\widetilde{M}_{j,\beta_{j}}^{(d,\overline{d})}(h_{i})\left((\widetilde{M}_{j,\beta_{j}}^{(d,\overline{d})}(h_{i}))^{L_{i}-L_{j,i}-1}-1\right)}{(\widetilde{M}_{j,\beta_{j}}^{(d,\overline{d})}(h_{i}))-1}$ and $\gamma^{(d)}=\frac{(1-\rho_{j,\beta_{j}}^{(d)})t_{i}(\widetilde{M}_{j,\beta_{j}}^{(\overline{d})}(h_{i}))^{L_{i}+1}}{\left[h_{i}-\Lambda_{j,\beta_{j}}^{(d)}(B_{j,\beta_{j}}^{(d)}(h_{i})-1)\right](\widetilde{M}_{j,\beta_{j}}^{(d)}(h_{i}))^{L_{j,i}+1}}$. Further, $\widetilde{M}_{j,\beta_{j}}^{(d)}(h_{i})=\frac{\alpha_{j,\beta_{j}}^{(d)}e^{\eta_{j,\beta_{j}}-h_{i}\tau}}{\alpha_{j,\beta_{j}}^{(d)}-h_{i}}$, $\widetilde{M}_{j,\beta_{j}}^{(\overline{d})}(h_{i})=\frac{\alpha_{j,\beta_{j}}^{(\overline{d})}e^{\eta_{j,\beta_{j}}-h_{i}\tau}}{\alpha_{j,\beta_{j}}^{(\overline{d})}-h_{i}}$, $\widetilde{M}_{j,\nu_{j}}^{(e)}(h_{i})=\frac{\alpha_{j,\nu_{j}}^{(e)}e^{\eta_{j,\nu_{j}}-h_{i}\tau}}{\alpha_{j,\nu_{j}}^{(e)}-h_{i}}$. This proves the statement of the Theorem.

Appendix F Sub-problems Optimization

In this section, we explain how each sub-optimization problem is solved.

F-1 Server-PSs Access Optimization

Given the bandwidth allocation weights, the cache placement, edge-cache window size, and the auxiliary variables, this sub-problem can be written as follows.

Input: $\boldsymbol{h}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$

Objective:            min $\left(\ref{eq:optfun}\right)$

s.t. (36)– (39), –(41)– (44)

                          var. $\widetilde{\boldsymbol{\pi}}$

In order to solve this problem, we use iNner cOnVex Approximation (NOVA) algorithm proposed in [15]. The key idea for this algorithm is that the non-convex objective function is replaced by suitable convex approximations at which convergence to a stationary solution of the original non-convex optimization is established. NOVA solves the approximated function efficiently and maintains feasibility in each iteration. The objective function can be approximated by a convex one (e.g., proximal gradient-like approximation) such that the first order properties are preserved [15], and this convex approximation can be used in NOVA algorithm.

Let $\widetilde{U_{q}}\left(\widetilde{\boldsymbol{\pi}},\boldsymbol{\widetilde{\pi}^{\nu}}\right)$ be the convex approximation at iterate $\boldsymbol{{\pi}^{\nu}}$ to the original non-convex problem $U\left(\widetilde{\boldsymbol{\pi}}\right)$, where $U\left(\widetilde{\boldsymbol{\pi}}\right)$ is given by (35). Then, a valid choice of $U\left(\widetilde{\boldsymbol{\pi}};\boldsymbol{{\pi}^{\nu}}\right)$ is the first order approximation of $U\left(\widetilde{\boldsymbol{\pi}}\right)$, e.g., (proximal) gradient-like approximation, i.e.,

[TABLE]

where $\tau_{u}$ is a regularization parameter. Note that all the constraints (36)– (39) are separable and linear in $\widetilde{\pi}_{i,j,k}$. The NOVA Algorithm for optimizing $\widetilde{\boldsymbol{\pi}}$ is described in Algorithm 1. Using the convex approximation $\widetilde{U_{\pi}}\left(\boldsymbol{\pi};\boldsymbol{{\pi}^{\nu}}\right)$, the minimization steps in Algorithm 1 are convex, with linear constraints and thus can be solved using a projected gradient descent algorithm. A step-size ($\gamma$) is also used in the update of the iterate $\widetilde{\boldsymbol{\pi}}^{\nu}$. Note that the iterates $\left\{\boldsymbol{\pi}^{\nu}\right\}$ generated by the algorithm are all feasible for the original problem and, further, convergence is guaranteed, as shown in [15] and described in lemma 6.

In order to use NOVA, there are some assumptions (given in [15]) that have to be satisfied in both original function and its approximation. These assumptions can be classified into two categories. The first category is the set of conditions that ensure that the original problem and its constraints are continuously differentiable on the domain of the function, which are satisfied in our problem. The second category is the set of conditions that ensures that the approximation of the original problem is uniformly strongly convex on the domain of the function. The latter set of conditions are also satisfied as the chosen function is strongly convex and its domain is also convex. To see this, we need to show that the constraints (37)–(41) form a convex domain in $\widetilde{\boldsymbol{\pi}}$ which is easy to see from the linearity of the constraints. Further details on the assumptions and function approximation can be found in [15]. Thus, the following result holds.

Lemma 6**.**

For fixed $\boldsymbol{h}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$, the optimization of our problem over $\widetilde{\boldsymbol{\pi}}$ generates a sequence of decreasing objective values and therefore is guaranteed to converge to a stationary point.

F-2 Auxiliary Variables Optimization

Given the probability distribution of the server-PSs scheduling probabilities, the bandwidth allocation weights, edge-cache window size, and the cache placement, this subproblem can be written as follows.

**Input: $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$ **

Objective:            min $\left(\ref{eq:optfun}\right)$

s.t. (36), (41)–(44),

                          var. $\boldsymbol{h}$

Similar to Server-PSs Access Optimization, this optimization can be solved using NOVA algorithm. The constraint (41) is linear in $\boldsymbol{h}$. Further, the next Lemma show that the constraints (42)– (44) are convex in $\boldsymbol{h}$, respectively.

Lemma 7**.**

The constraints (42)–(44) are convex with respect to $\boldsymbol{{h}}$.

Proof.

The proof is given in Appendix G. ∎

Algorithm 2 shows the used procedure to solve for $\boldsymbol{h}$. Let $\overline{U}\left(\boldsymbol{h};\boldsymbol{h^{\nu}}\right)$ be the convex approximation at iterate $\boldsymbol{h^{\nu}}$ to the original non-convex problem $U\left(\boldsymbol{h}\right)$, where $U\left(\boldsymbol{h}\right)$ is given by (35), assuming other parameters constant. Then, a valid choice of $\overline{U}\left(\boldsymbol{h};\boldsymbol{h^{\nu}}\right)$ is the first order approximation of $U\left(\boldsymbol{h}\right)$, i.e.,

[TABLE]

where $\tau_{h}$ is a regularization parameter. The detailed steps can be seen in Algorithm 2. Since all the constraints (41)– (44) have been shown to be convex in $\boldsymbol{h}$, the optimization problem in Step 1 of Algorithm 2 can be solved by the standard projected gradient descent algorithm.

Lemma 8**.**

For fixed $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{w}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$, the optimization of our problem over $\boldsymbol{h}$ generates a sequence of monotonically decreasing objective values and therefore is guaranteed to converge to a stationary point.

F-3 Bandwidth Allocation Weights Optimization

Given the auxiliary variables, the server access and PSs selection probabilities, edge-cache window size, and cache placement, this subproblem can be written as follows.

Input: $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{L}$, $\boldsymbol{\omega}$, and $\boldsymbol{h}$

Objective:            min $\left(\ref{eq:optfun}\right)$

s.t. (36)–(38),  (41)–(44),

                          var. $\boldsymbol{w}$

This optimization problem can be solved using NOVA algorithm. It is easy to notice that the constraints that exist in (36)– (38) are linear and thus convex with respect to $\boldsymbol{w}$. Further, the next two Lemmas show that the constraints (41)–(44), are convex in $\boldsymbol{w}$, respectively.

Lemma 9**.**

The constraints (41)–(44) are convex with respect to $\boldsymbol{{w}}$.

Proof.

The proof is given in Appendix G. ∎

Algorithm 3 shows the used procedure to solve for $\boldsymbol{w}$. Let ${U_{w}}\left(\boldsymbol{w};\boldsymbol{w^{\nu}}\right)$ be the convex approximation at iterate $\boldsymbol{w^{\nu}}$ to the original non-convex problem $U\left(\boldsymbol{w}\right)$, where $U\left(\boldsymbol{w}\right)$ is given by (35), assuming other parameters constant. Then, a valid choice of ${U_{w}}\left(\boldsymbol{w};\boldsymbol{w^{\nu}}\right)$ is the first order approximation of $U\left(\boldsymbol{w}\right)$, i.e.,

[TABLE]

where $\tau_{t}$ is a regularization parameter. The detailed steps can be seen in Algorithm 3. Since all the constraints have been shown to be convex, the optimization problem in Step 1 of Algorithm 3 can be solved by the standard projected gradient descent algorithm.

Lemma 10**.**

For fixed $\widetilde{\boldsymbol{\pi}}$ , $\boldsymbol{h}$, $\boldsymbol{\omega}$, and $\boldsymbol{L}$, the optimization of our problem over $\boldsymbol{w}$ generates a sequence of decreasing objective values and therefore is guaranteed to converge to a stationary point.

F-4 Cache Placement Optimization

Given the auxiliary variables, the server access and PS selection probabilities, edge-cache window size, and the bandwidth allocation weights, this subproblem can be written as follows.

Input: $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{h}$, $\boldsymbol{\omega}$ and $\boldsymbol{w}$

Objective:            min $\left(\ref{eq:optfun}\right)$

s.t. (36)– (39), (40), (42)– (44)

                          var. $\boldsymbol{L}$

Similar to the aforementioned Optimization sub-problems, this optimization can be solved using NOVA algorithm. Constraints (37)– (39), are linear in $\boldsymbol{L}$, and hence, form a convex domain. Also, Constraint (40) is relaxed to have it convex. Furthermore, the constraints (42)– (44) are convex as shown in the following Lemmas in this subsection.

Algorithm 4 shows the used procedure to solve for $\boldsymbol{L}$. Let ${U_{L}}\left(\boldsymbol{L};\boldsymbol{L^{\nu}}\right)$ be the convex approximation at iterate $\boldsymbol{L^{\nu}}$ to the original non-convex problem $U\left(\boldsymbol{L}\right)$, where $U\left(\boldsymbol{L}\right)$ is given by (35), assuming other parameters constant. Then, a valid choice of ${U_{L}}\left(\boldsymbol{L};\boldsymbol{L^{\nu}}\right)$ is the first order approximation of $U\left(\boldsymbol{L}\right)$, i.e.,

[TABLE]

where $\tau_{L}$ is a regularization parameter. The detailed steps can be seen in Algorithm 3. Since all the constraints have been shown to be convex in $\boldsymbol{L}$, the optimization problem in Step 1 of Algorithm 3 can be solved by the standard projected gradient descent algorithm.

Lemma 11**.**

For fixed $\boldsymbol{h}$, $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{\omega}$ and $\boldsymbol{w}$, the optimization of our problem over $\boldsymbol{L}$ generates a sequence of monotonically decreasing objective values and therefore is guaranteed to converge to a stationary point.

F-5 Edge-cache Window size Optimization

Given the server access and PS selection probabilities, the bandwidth allocation weights, the cache placement, and the auxiliary variables, this sub-problem can be written as follows.

Input: $\boldsymbol{h}$, $\boldsymbol{w}$, $\boldsymbol{\widetilde{\pi}}$, and $\boldsymbol{L}$

Objective:            min $\left(\ref{eq:optfun}\right)$

s.t. (36)–(39), (42)–(44)

                          var. $\boldsymbol{\omega}$

similarly, this optimization can be solved using NOVA algorithm. It is easy to show that Constraints (37)–(39), are convex in $\boldsymbol{\omega}$, and hence, form a convex domain. Further, the constraints (42)–(44) are convex as shown in Lemma 12.

Algorithm 4 shows the used procedure to solve for $\boldsymbol{\omega}$. Let ${U_{\omega}}\left(\boldsymbol{\omega};\boldsymbol{\omega^{\nu}}\right)$ be the convex approximation at iterate $\boldsymbol{\omega^{\nu}}$ to the original non-convex problem $U\left(\boldsymbol{\omega}\right)$, where $U\left(\boldsymbol{\omega}\right)$ is given by (35), assuming other parameters constant. Then, a valid choice of ${U_{\omega}}\left(\boldsymbol{\omega};\boldsymbol{\omega^{\nu}}\right)$ is the first order approximation of $U\left(\boldsymbol{\omega}\right)$, i.e.,

[TABLE]

where $\tau_{\omega}$ is a regularization parameter. The detailed steps can be seen in Algorithm 3. Since all the constraints have been shown to be convex in $\boldsymbol{\omega}$, the optimization problem in Step 1 of Algorithm 3 can be solved by the standard projected gradient descent algorithm.

Lemma 12**.**

For fixed $\boldsymbol{h}$, $\widetilde{\boldsymbol{\pi}}$, $\boldsymbol{\omega}$ and $\boldsymbol{w}$, the optimization of our problem over $\boldsymbol{\omega}$ generates a sequence of monotonically decreasing objective values and therefore is guaranteed to converge to a stationary point.

Proof.

The proof is provided in Appendix G. ∎

Appendix G Proof of Results in Appendix F

G-A Proof of Lemma 7

The constraints (42)–(44) are separable for each $h_{i}$ and due to symmetry of the three constraints it is enough to prove convexity of $E(h)=\sum_{f=1}^{r}\pi_{f,j}q_{f,j,\beta_{j}}\lambda_{f}e^{-\lambda_{i}\omega_{i}}\left(\frac{\alpha}{\alpha-h_{i}}\right)^{L_{f}-L_{j,f}}-\left(\Lambda_{j,\beta_{j}}+h_{i}\right)$, assuming that the edge router $\ell$ is unfold, without loss of generality. Thus, it is enough to prove that $E^{\prime\prime}(h)\geq 0$. We further note that it is enough to prove that $D^{\prime\prime}(h)\geq 0$, where $D(h)=\left(\frac{\alpha}{\alpha-t_{i}}\right)^{L_{f}-L_{j,f}}$. This follows since

[TABLE]

G-B Proof of Lemma 9

The constraint (42)–(44) are separable for each $\alpha_{j,\beta_{j}}^{(d)}$, $\alpha_{j,\beta_{j}}^{(c)}$ and $\alpha_{j,\nu_{j}}^{(e)}$, respectively. Note that we omit the subscript $\ell$ for simplicity, w.l.o.g. Thus, it is enough to prove convexity of the following three equations

[TABLE]

[TABLE]

for $h<\alpha_{j,\beta_{j}}^{(d)}$, $h<\alpha_{j,\beta_{j}}^{(c)}$, and $h<\alpha_{j,\nu_{j}}^{(e)}$, respectively. Since there is only a single index $j$, $\beta_{j}$, and $\nu_{j}$, here, we ignore the subscripts and superscripts for the rest of this proof and prove for only one case due to the symmetry. Thus, it is enough to prove that $E_{1}^{\prime\prime}(\alpha)\geq 0$ for $h<\alpha$. We further note that it is enough to prove that $D_{1}^{\prime\prime}(\alpha)\geq 0$, where $D_{1}(\alpha)=\left(1-\frac{h}{\alpha}\right)^{L_{j,i}-L_{i}}$. This holds since,

[TABLE]

Appendix H Key notations used in this paper

The key used notation in this paper are shown in Table III.

Appendix I Mean Stall Duration

In this section, a bound for the mean stall duration, for any video file $i$, is provided. Since probabilistic scheduling is one feasible strategy, the obtained bound is an upper bound to the optimal strategy.

Using equation (76), the stall duration for the request of file $i$ from $\beta_{j}$ queue, $\nu_{j}$ queue and server $j$, $\Gamma^{(i,j,\beta_{j},\nu_{j})}_{U}$ is given as

[TABLE]

An exact evaluation for the play time of segment $L_{i}$ is hard due to the dependencies between $\mathcal{F}_{i,j,\nu_{j},\beta_{j},z}$ (i.e., equation (74)) random variables for different values of $j$, $\nu_{j}$, $\beta_{j}$ and $z$, where $z\in{(1,2,...,L_{i}+1)}$. Hence, we derive an upper-bound on the playtime of the segment $L_{i}$ as follows. Using Jensen’s inequality [44], we have for $g_{i}>0$,

[TABLE]

Thus, finding an upper bound on the moment generating function for $T_{i}^{(L_{i})}$ will lead to an upper bound on the mean stall duration. Hence, we will now bound the moment generating function for $T_{i}^{(L_{i})}$. Using equation (88), we can show that

[TABLE]

Substituting (100) in (99), the mean stall duration is bounded as follows.

Theorem 3**.**

The mean stall duration time for file $i$ is bounded by

[TABLE]

*for any $t_{i}>0$, $\rho_{j,\nu_{j},\ell}^{(e)}<1$, $\rho_{j,\beta_{j},\ell}^{(\overline{d})}<1$ , and $\rho_{j,\nu_{j},\ell}^{(e)}<1$  and

the involved MGFs exist, $\forall j,\nu_{j},\beta_{j}$.*

We note that for the scenario, where the files are downloaded rather than streamed, a metric of interest is the mean download time. This is a special case of our approach when the number of segments of each video is one, or $L_{i}=1$. Thus, the mean download time of the file follows as a special case of Theorem 3.

Appendix J Online Algorithm for Edge-cache Placement

We note that for the setup of the edge cache, we assumed that the edge-cache has a capacity of $C_{e,\ell}$ seconds (ignoring the index of the edge cache). However, in the caching policy, we assumed that a file $f$ is removed from the edge cache $\ell$ if it has not been requested in the last $\omega_{f,\ell}$ seconds. In the optimization, we found the parameters $\omega_{f,\ell}$, such that the cache capacity is exceeded with probability less than $\epsilon_{\ell}$. However, this still assumes that it is possible to exceed the cache capacity some times. This is, in practice, not possible. Thus, we will propose a mechanism to adapt the decision obtained by the optimization formulation so as to never exceed the edge cache capacity.

When a file $i$ is requested, the last request of file $i$ is first checked. If it has not been requested in the last $\omega_{i,\ell}$ seconds, it is obtained from the CDN. In order to do that, the space of the file is reserved in the edge-cache. If this reservation exceeds the capacity of the edge-cache, certain files have to be removed. Any file $f$ that has not been requested in the last $\omega_{f,\ell}$ seconds is removed from the cache. If, even after removing these files, the space in the edge-cache is not enough for placing file $i$ in the edge-cache, more files must be removed. Assume that $\mathcal{H}$ is the set contains all files in the edge-cache, and $t_{f,l_{t}}$ is the last time file $f$ has been requested. Then, if another file needs to be removed to make space for the newly requested file, the file $\text{argmin}_{f\in\mathcal{H}}(t_{f,l_{t}}+\omega_{f,\ell}-t_{i})$ is removed. This continues till there is enough space for the new incoming file. Note that multiple files may be removed to make space for the incoming file, depending on the length of the new file. This is similar concept to LRU where a complete new file is added in the cache, and multiple small files may have to be removed to make space. The key part of the online adaptation so as not to violate the edge-cache capacity constraint is illustrated in Figure 8. This flowchart illustrates the online updates for an edge-cache when a file $i$ is requested at time $t_{i}$.

Appendix K Edge-cache Performance and further Evaluation

Convergence of the proposed algorithm: Figure 9 shows the convergence of our proposed SDTP algorithm, which alternatively optimizes the weighted SDTP of all files over scheduling probabilities $\widetilde{\boldsymbol{\pi}}$, auxiliary variables $\boldsymbol{t}$, bandwidth allocation weights $\boldsymbol{w}$, cache server placement $\boldsymbol{L}$, and window-size $\omega_{i}$. We see that for $r=500$ video files of size $600$s with $m=5$ cache storage nodes, the weighted stall duration tail probability converges within a few iterations.

Effect of scaling up the bandwidth of the cache servers and datacenter: The effect of increasing the server bandwidth on the weighted SDTP is plotted in Figure 10. Intuitively, increasing the storage node bandwidth will increase the service rate of the storage nodes by assigning higher bandwidth to the users, thus, reducing the weighted SDTP.

Effect of the bound percentage $\epsilon$ in the SDTP: Figure 11 plots the weighted SDTP versus $\epsilon$, i.e., probability that the cache size is exceeded. We see that the SDTP increases significantly with an increase in $\epsilon$. This is because as $\epsilon$ increases, there are more edge capacity constraint violations and the the online adaptations may not remain the optimal choice.

In the following figures, a trace-based implementation is performed, where the video ID, time requests, video lengths, etc. are obtained from one-week traces of a production system from the major service provider in the US. We note that the arrival process is not Poisson in this case, while the proposed approach still outperform the considered baseline approaches.

Effect of the arrival rates on the TTFC: Figure 12 shows the effect of different video arrival rates on the TTFC for different-size video lengths. The different sizes for video files are obtained from real traces of a major video service provider. We compared our proposed online algorithm with the analytical offline bound and LRU-based (explained in Section IV, B) policies. We see that the TTFC increases with arrival rates, as expected, however, since the TTFC is more significant at high arrival rates, we notice a significant improvement in the download time of the first chunk by about 60% at the highest arrival rate in Figure 12 as compared to the LRU policy.

Effect of arrival Rates on the MSD: The effect of different video arrival rates on the mean stall duration for different-size video length is captured in Figure 13. We compared our proposed online algorithm with five baseline policies and we see that the proposed algorithm outperforms all baseline strategies for the QoE metric of mean stall duration. Thus, bandwidth, size of the time-window, access and placement of files in the storage caches are important for the reduction of mean stall duration. Further, obviously, the mean stall duration increases with arrival rates, as expected. Since the mean stall duration is more significant at high arrival rates, we notice a significant improvement in mean stall duration (approximately 15s to about 5s) at the highest arrival rate in Figure 13 as compared to the LRU policy

Effect of edge-cache capacity: We study the miss-rate (percentage of how many video file requests are not served from the edge-cache) performance of the edge-cache. Clearly, the miss-rate decreases with the increasing size of the capacity of the edge-cache. However, when the edge-cache capacity is approximately 35% of the entire video sizes, the miss-rate is around 20%. Further, adaptSize policy does not neither optimize the time to live window of files $\omega_{i}$’s nor intelligently incorporate the arrival rates in adding/evicting the video files, and thus its performance becomes less sensitive to varying the cache size. The variant versions of LRU (qLRU with $q=0.67$, kLRU and kRandom with $k=6$) obtain closer performance compared to that of the basic LRU where kLRU performs that best among them as it somehow maintains a window (k-requests) for admitting a file into the cache and adapts LRU policy in the eviction process.

Appendix L Joint Mean-Tail Optimization

We wish to jointly minimize the two QoE metrics (MSD and SDTP) over the choice of server-PSs scheduling, bandwidth allocation, edge-cache window-size and auxiliary variables. Since this is a multi-objective optimization, the objective can be modeled as a convex combination of the two QoE metrics.

The first objective is the minimization of the mean stall duration, averaged over all the file requests, and is given as $\sum_{i,\ell}\frac{\lambda_{i,\ell}}{\overline{\lambda}}\,\mathbb{E}\left[\Gamma^{\left(i,\ell\right)}\right]$. The second objective is the minimization of stall duration tail probability, averaged over all the video file requests, and is given as $\sum_{i,\ell}\frac{\lambda_{i,\ell}}{\overline{\lambda}}\,{\text{Pr}}\left(\Gamma^{(i,\ell)}\geq x\right)$. Using the expressions for the mean stall duration and the stall duration tail probability, respectively, optimization of a convex combination of the two QoE metrics can be formulated as follows.

[TABLE]

Clearly, the above optimization problem is non-convex in all the parameters jointly. This is can be easily seen in the terms which are product of the different variables. Since the problem is non-convex, we propose an iterative algorithm to solve the problem. This algorithm performs an alternating optimization over the different aforementioned dimensions, such that each sub-problem is shown to have convex constraints and thus can be efficiently solved using NOVA algorithm [15]. The subproblems are explained in detailed in Appendix F.

Mean-Tail tradeoff: There is a tradeoff between the MSD and SDTP. Hence, we now investigate this tradeoff in order to get a better understanding of how this traddeoff can be compromised. To do so, we vary $\theta$ in the above optimization problem to get a tradeoff between MSD and SDTP. Intuitively, if the mean stall duration decreases, the stall duration tail probability also reduces, as depicted in Figure 15. Therefore, a question arises whether the optimal point for decreasing the mean stall duration and the stall duration tail probability is the same? Based on our real video traces, we answer this question in negative since we find that at the desgin values that optimize the mean stall duration, the stall duration tail probability is 10+ times higher as compared to the optimal stall duration tail probability. Similarly, the optimal mean stall duration is 7 times lower as compared to the mean stall duration at the design values that optimizes the stall duration tail probability. As a result, an efficient tradeoff point between the two QoE metrics can be chosen based on the point on the curve that is appropriate for the clients.

Appendix M Extension to Different Quality levels

In this section, we show how our analysis can be extended to cover the scenario when the video can be streamed at different quality levels. We assume that each video file is encoded to different qualities, i.e., $Q\in\{1,2,\cdots,V\}$, where $V$ is the number of possible choices for the quality level. The $L_{i}$ chunks of video file $i$ at quality $Q$ are denoted as $G_{i,Q,1},\cdots,G_{i,Q,L_{i}}$. We will use a probabilistic quality assignment strategy, where a chunk of quality $Q$ of size $a_{Q}$ is requested with probability $b_{i,Q}$ for all $Q\in\{1,2,\cdots,V\}$. We further assume all the chunks of the video are fetched at the same quality level. From Section III-D, we can show that the for a file of quality $Q$ requested from edge-router $\ell$, we choose server $j$ with probability $\pi_{i,j,\ell}^{(Q)}$. Further, we can show that the aggregate arrival rate at $PS^{(d,j)}_{\beta_{j}}$, $PS^{(\overline{d},j)}_{\beta_{j},\ell}$, and $PS^{(e,j)}_{\nu_{j},\ell}$, denoted as $\Lambda^{(d)}_{j,\beta_{j}}$, $\Lambda^{(\overline{d})}_{j,\beta_{j},\ell}$,and $\Lambda^{(e)}_{j,\nu_{j},\ell}$, respectively are given as follows.

[TABLE]

Similarly, we can define

[TABLE]

for all $\beta_{j}$, $\nu_{j}$, $Q$, and $\ell$. Note that $\alpha_{j}^{(.,Q)}=\alpha_{j}^{(.)}/a_{\ell}$ where $alpha_{j}^{(.)}$ is a constant service time parameter when $a_{\ell}=1$. We further define the moment generating functions of the service times of $PS^{(d,j,Q)}_{\beta_{j}}$, $PS^{(\overline{d},j,Q)}_{\beta_{j},\ell}$, and $PS^{(e,j,Q)}_{\nu_{j},\ell}$ as $M^{(d,Q)}_{j,\beta_{j}}$, $M^{(\overline{d},Q)}_{j,\beta_{j},\ell}$, and $M^{(\overline{d},Q)}_{j,\nu_{j},\ell}$, which are defined as follows.

[TABLE]

where $\eta_{j,.}^{(.,Q)}=\eta_{j,.}^{(.)}\times a_{\ell}$ where $eta_{j,.}^{(.)}$ is a constant time shift parameter when $a_{\ell}=1$. Following the same analysis as in Section IV, it is easy to show that the stall duration for the request of file $i$ at quality $Q$ from $\beta_{j}$ queue, $\nu_{j}$ queue and server $j$, if not in the edge-cache, i.e., $\Gamma^{(i,j,\beta_{j},\nu_{j},Q)}_{U}$ is given as

[TABLE]

This expression is used to derive tight bounds on the QoE metrics. By simplifications and some algebraic manipulation,the following theorems can be derived.

Theorem 4**.**

The mean stall duration for video file $i$ streamed with quality $Q$ requested through edge router $\ell$ is bounded by

[TABLE]

where:

[TABLE]

and $\widetilde{c}$, $\widetilde{a}$, and $\overline{a}$ are defined earlier in Section IV, e.g., equation (19).

Theorem 5**.**

The stall duration tail probability for video file $i$ requested at quality $Q$ and through edge router $\ell$ is bounded by

[TABLE]

for $\rho_{j,\beta_{j}}^{(d)}<1$, $\rho_{j,\beta_{j},\ell}^{(\overline{d})}<1$, $\rho_{j,\nu_{j},\ell}^{(e)}<1$, where the auxiliary variables in the statement of the Theorem are similarly defined as those in equations (23)-(34).

Having derived the MSD and SDTP, one can formulate a constrained optimization problem to jointly optimize a convex combination of all QoE metrics as follows

[TABLE]

To solve this porblem, we still have to use alternating minization based approaches since the problem is not jointly convex in all the optimized parametes. Thus, we propose an iterative algorithm (similar to that explained in F) to solve the problem. The proposed algorithm divides the problem into sub-problems that optimizes one variable at a time while fixing the other. We refer the interested reader to [45] for detailed treatment of this problem.

We also note that similar methodology can be used to handle the scenarios where the chunks have different sizes. If the video files have different chunk size (in MB), our analysis can be easily extended to handle such cases as follows. Since a video file has different chunk sizes, the service time will be different from one chunk to another. However, one can still get the MGF of the service time in a similar fashion to those in (114)-(116). Thus, the service time of a chunk will be related to its size. For example, for a chunk indexed by $\kappa$ and requested from the PS $PS^{(d,j,\kappa)}_{\beta_{j}}$, the MGF of the servce time will be $M_{j,\beta_{j}}^{(d,\kappa)}$. Hence, similar formula to that in (118) for the stall duration under different sizes for the chunks can be obtained.