Event-triggered Online Proactive Network Association to Mobile Edge Computing for IoT

TL;DR

This paper introduces an online proactive network association method for IoT that minimizes task delay and energy consumption in mobile edge computing environments, using Markov decision processes and Lyapunov optimization.

Contribution

It proposes a novel event-triggered, mobility-aware proactive network association model with a two-stage online algorithm that does not require prior knowledge of random event distributions.

Findings

The TOPNA algorithm effectively reduces task delay.

The approach manages energy consumption under system uncertainties.

Simulation results confirm the algorithm's effectiveness.

Abstract

Ultra-low latency communication for mobile machines emerges as a critical technology in Internet of Things (IoT). Proactive network association has been suggested to support ultra-low latency communication with the assistance of mobile edge computing. To resolve system dynamics and uncertainty, in this paper, an online proactive network association is proposed to minimize average task delay while considering time-average energy consumption constraints. Under distributed computing and networking environments, we formulate an event-triggered proactive network association model by semi-Markov task states and independent identically distributed (i.i.d.) random events. Then we facilitate the mobility-aware anticipatory network association to predictively consider handover effects caused by the mobility. Based on the Markov decision processes (MDP) and Lyapunov optimization, the two-stage online proactive network association (TOPNA) decision algorithm is proposed without the knowledge nor distribution of random events. Simulation results exhibit the effectiveness of the proposed algorithm.

Figures

Equations

$$\mathcal{A}(r)=\left\{j|\Vert \ell (r)-{\ell }_j\Vert \le Ra,\forall j\in \mathcal{M}\right\},$$

$$\begin{array}[]{l}d_{h}({{\bf{s}}}(r),{{\bf{w}}}(r),{a}(r))\\ =\left\{{\begin{array}[]{*{20}{c}}{0,}&{{a}(r)={m}(r)}\\ {C,}&{{a}(r)\neq{m}(r),}\end{array}}\right.\end{array}$$

$$d_{tr}(\mathbf{s}(r),\mathbf{w}(r),a(r))=\frac{\alpha (r)}{c_{a(r)}(r)},$$

$$d_c(\mathbf{s}(r),\mathbf{w}(r),a(r))=\frac{\alpha (r)\epsilon }{f_{a(r)}(r)}.$$

$$\begin{array}{rl}d(r)& \triangleq d_h(\mathbf{s}(r),\mathbf{w}(r),a(r))+d_{tr}(\mathbf{s}(r),\mathbf{w}(r),a(r))\\ & +d_c(\mathbf{s}(r),\mathbf{w}(r),a(r)),\end{array}$$

$$e(\mathbf{s}(r),\mathbf{w}(r),a(r))=\frac{p_{\text{tx}}\alpha (r)}{c_{a(r)}(r)},$$

$$\begin{array}[]{l}{C}({{\bf{s}}}(r),{{\bf{w}}}(r),{a}(r),{{\bf{s}}}(r+1))=\left\{{\begin{array}[]{*{20}{c}}{d_{tr}(r)+d_{c}(r),}&{{{a}}(r)={m}(r),{{a}}(r)\in{{\cal A}}(r+1)}\\ {d_{tr}(r)+d_{c}(r)+d_{h}(r),}&{{{a}}(r)={m}(r),{{a}}(r)\notin{{\cal A}}(r+1)}\\ {d_{tr}(r)+d_{c}(r)+d_{h}(r),}&{{{a}}(r)\neq{m}(r),{{a}}(r)\in{{\cal A}}(r+1)}\\ {d_{tr}(r)+d_{c}(r)+2d_{h}(r),}&{{{a}}(r)\neq{m}(r),{{a}}(r)\notin{{\cal A}}(r+1)}\,.\end{array}}\right.\end{array}$$

$$\overline{d}(R)=\frac{1}{R}{\sum }_{r=0}^{R-1}\mathbb{E}[d(r)].$$

$$\frac{\overline{e}}{\overline{T}}=\underset{R\to \infty }{\lim }\frac{\frac{1}{R}{\sum }_{r=0}^{R-1}\mathbb{E}[e(r)]}{\frac{1}{R}{\sum }_{r=0}^{R-1}\mathbb{E}[T(r)]},$$

$$\text{P}:$$

$$\text{C1}:\frac{\overline{e}}{\overline{T}}\le \beta ;\text{C2}:a(r)\in \mathcal{A}(r),\forall r\in \mathcal{R},$$

$$\begin{array}{rl}& \mathrm{Pr}\{\mathbf{s}(r+1)|\mathbf{s}(r),a(r)\}\\ =& \mathrm{Pr}(m(r+1)|m(r),a(r))\cdot \mathrm{Pr}(\ell (r+1)|\ell (r)),\end{array}$$

$$\Pr({m}(r+1)|{m}(r),{a}(r))=\left\{{\begin{array}[]{*{20}{c}}{1,}&{{\textrm{if}}\;{m}(r+1)={{a}}(r)}\\ {0,}&{{\textrm{otherwise}}.}\end{array}}\right.$$

$$\begin{array}{rl}& d^{{}^{\prime }}(\mathbf{s}(r),\mathbf{w}(r),a(r))\\ & ={\sum }_{\mathbf{s}(r+1)\in \mathcal{S}}C\left(\mathbf{s}(r),\mathbf{w}(r),a(r),\mathbf{s}(r+1)\right)\\ & \cdot Pr(\mathbf{s}(r+1)|\mathbf{s}(r),a(r)).\end{array}$$

$$\text{P1}:$$

$$\text{C1}:\frac{\overline{e}}{\overline{T}}\le \beta ;\text{C2}:a(r)\in \mathcal{A}(r),\forall r\in \mathcal{R}.$$

$$q_{mn}(r)=1(\mathbf{s}(r)=m)\mathrm{Pr}(\mathbf{s}(r+1)=n|\mathbf{s}(r)=m,a(r)),$$

$$\text{P2}:$$

$$\text{C1}:\frac{\overline{e}}{\overline{T}}\le \beta ;\text{C2}:{\sum }_{m=1}^S\overline{q_{sm}}={\sum }_{n=1}^S\overline{q_{ns}},\forall s\in \mathcal{S};$$

$$\text{C3}:\mathbf{s}(r)\in \mathcal{S},\forall r\in \mathcal{R};\text{C4}:a(r)\in \mathcal{A}(r),\forall r\in \mathcal{R};$$

$$\text{C5}:\mathbf{s}(r)\text{is}\text{selected}\text{independently}\text{of}\mathbf{w}(r),$$

$$E(r+1)=\max \{E(r)+e(r)-\beta T(r),0\}.$$

$$G_s(r+1)=G_s(r)+1(\mathbf{s}(r)=s)-{\sum }_{n=1}^S{q}_{ns}(r).$$

$$L(r)=\frac{1}{2}E(r{)}^2+\frac{1}{2}{\sum }_{s=1}^S{G}_s{(r)}^2,$$

$${\Delta }_V(r)=\Delta (r)+V\mathbb{E}[d^{{}^{\prime }}(r)|\Theta (r)],$$

$${\Delta }_V(r)\le B+V\mathbb{E}[d^{{}^{\prime }}(r)|\Theta (r)]+E(r)\mathbb{E}[e(r)-\beta T(r)|\Theta (r)]$$

$$+{\sum }_{s=1}^S{G}_s(r)\mathbb{E}[1(\mathbf{s}(r)=s)-\mathrm{Pr}(s|\mathbf{s}(r),a(r))|\Theta (r)],$$

$${\hat{e}}_s(r)=\frac{1}{W}\sum _{w=1}^W{\min }_{a^{(w)}(r)\in \mathcal{A}(r)}$$

$${\mathbf{w}}^{(w)}(r),a^{(w)}(r))|\Theta (r)],$$

$$\underset{a(r)\in \mathcal{A}(r)}{\min }[V{d}^{{}^{\prime }}(r)+E(r)(e(r)-\beta (r)T(r))$$

$$+{\sum }_{s=1}^S{G}_s(r)(1(\mathbf{s}(r)=s)-Pr(s|\mathbf{s}(r),a(r)))].$$

$${\mathrm{e}}_s^{\ast }=\frac{\overline{1({\mathbf{s}}^{\ast }(r)=s)e(r)}}{\overline{1({\mathbf{s}}^{\ast }(r)=s)}},$$

$${\mathrm{T}}_s^{\ast }=\frac{\overline{1({\mathbf{s}}^{\ast }(r)=s)T(r)}}{\overline{1({\mathbf{s}}^{\ast }(r)=s)}},$$

$${\mathrm{Pr}}_{sm}^{\ast }=\frac{\overline{1({\mathbf{s}}^{\ast }(r)=s)q_{sm}(r)}}{\overline{1({\mathbf{s}}^{\ast }(r)=s)}},$$

Full text

I Introduction

With the rapid development of Internet of Things (IoT) applications, such as intelligent transportation, smart city and autonomous vehicles, etc. Ultra-low latency communication for mobile machines is critical to achieve system reliability [1]. For example, real-time vision analysis of surrounding road environment perceived by automatic vehicles to avoid potential traffic accidents. In recent years, mobile edge computing (MEC) has emerged to enable mobile applications due to its potential of achieving low delay and saving energy consumption through pushing computing capabilities to network edges [2]. Proactive network association therefore plays an important role to support ultra-low latency or delay-sensitive services under the assistance of MEC or fog network [3].

Different from traditional cloud computing with stable dedicated wired connection, mobile machines encounter highly dynamic and stochastic network environment, such as time-varying channel quality and unpredictable available server’s computing resource. However, most existing researches on network association ignore the stochasticity and unpredictability nature of mobile machines in such a network architecture. For example, in [4, 5], combinatorial optimization problem are formulated to myopically maximize transmission rates, which can be solved by distributed decomposition methods. In [6], a comprehensive survey of game theoretic approaches on network association is presented to choose the maximization of myopic utility between different strategies.

Some recent works have made some efforts to maximize long-term reward of network association in order to deal with the stochasticity of network environments. In [7, 8], Markov decision processes (MDP) are proposed to maximize the total amount of network association reward in the long run, which can be solved by linear programming. In [9, 10], Lyapunov optimization technologies are leveraged to optimize time-averaged energy consumption and network throughput accordingly by transforming stochastic optimization into deterministic optimization over timeslots.

However, the aforementioned studies treat decision-making of network association based on the discrete timestep or slotted structure with fixed and equal decision intervals, which violates the practical scenarios of operating mobile machines that the network association depending on the task completion duration [11]. Moreover, those investigations assume that the stochasticity of network environment satisfies the Markov dynamics or independent and identically distributed (i.i.d.) properties separately, which neglects the existence of both Markov property states (i.e., mobility trajectory and network association) and i.i.d. environment nature (i.e., channel quality and computing resource) in the network association problem, to over-simplify the low-latency operation and system reliability.

Facilitated by machine learning and autonomous operations by edge devices or smart machines/agents, proactive network association is critical in distributed computing scenarios, such as automatic driving and emergency rescue, by eliminating reliance on centralized decision-making entity [12]. In this paper, we develop the online proactive network association decision based on event-triggered distributed computing environment, to minimize the average task delay subjected to time-averaged energy consumption for mobile machines in IoT, particularly those mobile machines of artificial intelligence. The main contributions of this work are as follows:

• •

Different from the tradition network association model based on slotted structure, we develop an event-triggered proactive network association decision model under distributed computing environment by taking semi-Markov task states and i.i.d. random events into account.

• •

We extend the above model to the mobility-aware anticipatory network association decision problem by considering the further impacts on handover triggered by mobility.

• •

Based on the MDP and Lyapunov optimization theory, we propose the two-stage online proactive network association (TOPNA) decision algorithm. Simulation results show the effectiveness of our proposed algorithm.

II System Model

The network association system in consideration consists of a mobile machine and $M$ small base stations (SBSs), where the corresponding MEC server is deployed at each SBS, and the mobile machine can make proactive network association decision to offload or distribute the computation tasks to the MEC servers for processing. Let $\mathcal{M}=\{1,2,\cdots,M\}$ collect the indexes for $M$ MEC servers. The system model in this paper can be illustrated in Fig. 1.

II-A Computational Task Model

The mobile machine generates $R$ computing tasks to be offloaded during the movement over time. Let ${\cal R}=\{0,1,\ldots,R-1\}$ denote the set of computational task indexes. For each task $r\in\mathcal{R}$, we define a doublet $\mathcal{X}\buildrel\Delta\over{=}[{\alpha}(r),{\varepsilon}{\alpha}(r)]$ to parameterize the mobile machine’s computation task model, where ${\alpha}(r)$ is the input data size of $r$-th task and $\varepsilon$ represents the computation intensity that indicates the CPU cycles required to process a single computation task. Without loss of generality, we assume that the task size ${\alpha}(r)$ satisfies i.i.d. with unknown probability distribution function. But the task size is constrained by the minimum $A_{\min}$ and the maximum $A_{\max}$.

II-B Network Model

The set of candidate association servers for the $r$-th task of the mobile machine can be defined as

[TABLE]

where ${\ell}(r)$ is the position corresponding to the $r$-th task generated by the mobile machine, ${\ell}_{j}$ indicates the deployment location of server $j$ and $Ra$ is the coverage radius of each server.

Let $c_{j}(r)\in[c_{j}^{\min},c_{j}^{\max}]$ denote the channel capacity between the mobile machine and the MEC server $j$ during the processing of $r$-th task, where $c_{j}^{\min}$, $c_{j}^{\max}$ represent the minimum and the maximum capacity, respectively. Moreover, i.i.d. flat block fading channels are assumed, that is, the channel capacity remains unchanged during the same task processing and varies between different tasks. Similarly, we assume that the CPU-cycle frequency allocated by the MEC server $j$ satisfies i.i.d. stochastic process with the minimum $f_{j}^{\min}$ and the maximum $f_{j}^{\max}$, that is, $f_{j}(r)\in[f_{j}^{\min},f_{j}^{\max}]$.

II-C Task Processing Model

For the computation task $r\in\mathcal{R}$ of the mobile machine, we introduce the following two definitions:

• •

Semi-Markov states: ${{\bf{s}}}(r)=[{\ell}(r),{m}(r)]$ is the current state of task $r$, including the machine location ${\ell}(r)$ and the previous machine association ${m}(r)$, which can take values from the discrete state space $\mathcal{S}$ with cardinality $\mathrm{S}$. We assume that the task state transfers according to the Markov chain and operates in continuous time.

• •

Random events: ${{\bf{w}}}(r)=[{\alpha}(r),{c_{j}}(r),{f_{j}}(r),\forall j\in{\cal A}(r)]$ is defined as random events for task $r$. Each component of ${{\bf{w}}}(r)$ is i.i.d. over different tasks with an unknown probability distribution.

Note that the above definitions are given from the event-triggered perspective, that is, the event-triggering of network association decision depends on the completion time of computational task, which is different from the traditional slot-based structure of network association model with fixed equal decision intervals, as depicted in Fig. 2. The network handover occurs when the network association $a(r)$ is different from the previous one $m(r)$. Then the handover delay can be given by

[TABLE]

where $C$ is the total constant cost of signaling overhead and task migration caused by one handover.

After the association decision ${a}(r)$ is selected, the transmission delay of the computing task offloaded from the mobile machine to the server $a(r)$ can be expressed as

[TABLE]

then the task computation delay processed by the server $a(r)$ can be given by

[TABLE]

According to (2) - (4), we can define the total processing delay of task $r$ (i.e., task delay) as follows

[TABLE]

Moreover, the energy consumption of machine caused by the uplink transmission for computing task can be given by

[TABLE]

where $p_{\rm{tx}}$ is the transmit power of the mobile machine.

According to the above definition, we can define the task frame size as ${T}(r)\buildrel\Delta\over{=}{d}(r)$, which means that the adjacent decision interval is equal to the processing delay of previous task under the principle of first-in-first-out (FIFO) execution.

III Two-stage Online Proactive network Association decision

We first define the frame average of task processing delay (i.e., the average task delay) as follows

[TABLE]

Similarly, the frame average of energy consumption and task frame size can be defined as ${\overline{e}}(R)$, ${\overline{T}}(R)$, respectively. Then the time average of energy consumption can be given by

[TABLE]

where we rewrite $e(r)={e}({{\bf{s}}}(r),{{\bf{w}}}(r),{a}(r))$ for simplicity of notation, and we assume that the above stochastic processes ${d}(r)$, ${e}(r)$ and ${T}(r)$ converge to ${\overline{d}}$, ${\overline{e}}$ and ${\overline{T}}$ as the number of computation tasks $R\rightarrow\infty$. Then we can formulate P to minimize the average task delay as

[TABLE]

where $\beta$ is a given constant to specify the upper bound of time-averaged energy consumption constraint. Note that the above optimization problem is difficult to solve due to the existences of semi-Markov task state and random events. More specifically, the traditional MDP approach is inapplicable due to the unknown state transition probability and the curse of dimensionality caused by random events with unknown probability distribution. Furthermore, this problem is not a standard stochastic optimization due to the coupling introduced by the semi-Markov task states.

III-A Semi-Markovian Model and Mobility-aware Anticipatory Association

We first introduce a semi-Markovian model to precisely describe the behavior of task state. The task state transition probability can be derived as follows

[TABLE]

where the conditional transition probability of network association satisfies

[TABLE]

Considering the possibility that the mobile machine’s future mobility trajectory triggers a potential network handover, we define the task delay cost as given by (12). Then the expected task delay cost can be given by

[TABLE]

As a result, we construct the mobility-aware anticipatory network association problem P1 as follows

[TABLE]

III-B The Calculation of Optimal Proactive Network Association Parameters

Considering the coupling of task state, we first treat it as the decision variable rather than an observation of the environment to find the optimal parameters of proactive network association decision [13]. We introduce the following attribute as the observation during task frame, which suggests

[TABLE]

where $1({\mathbf{s}}(r)=m)$ is an indicator function. Then we can reformulate optimization problem as follows

[TABLE]

where constraints C1, C3 and C4 are self-explanatory. C2 ensures the global balance of state transition, which specifies that the frame average of task state in $s$ is equal to the frame average of transitioning to state $s$, Furthermore, C5 ensures that the decision variable selection of the task state should follow the principle of independence from random events.

According to the Lyapunov optimization theory [14], we define a virtual queue of energy consumption deviation, which can be evolved as

[TABLE]

Similarly, we can define virtual queue $G_{s}(r)$ to ensure the frame average equality constraint C2, which is updated by

[TABLE]

Then the Lyapunov function can be defined as

[TABLE]

and the drift-plus-penalty function can be given by

[TABLE]

where ${\Theta}(r)=\{{E}(r),G_{1}(r),G_{2}(r),...,G_{S}(r)\}$ is the collection of all queue backlogs at task frame $r$, and $\Delta(r)\triangleq\mathbb{E}[L(r+1)-L(r)|\Theta(r)]$ is the conditional Lyapunov drift. Moreover, $V$ is a predefined Lyapunov parameter. Then the upper bound of (20) can be derived in the following lemma.

Lemma 1

At task frame $r$, the drift-plus-penalty function (20) satisfies the following inequality:

[TABLE]

where $B$ is a finite constant and can be defined in the proof.

The proof is given in Appendix. Inspired by the maximum weight learning algorithm developed in [15], we use $W$ latest past samples of random events to evaluate the expectation value in task state $s$ instead of the exact probability distribution knowledge, which can be given by

[TABLE]

where $p({{\bf{s}}}(r)=s,{\bf{w}}^{(w)}(r),{a}^{(w)}(r))$ is the simplified expression of the right-hand side of (21) after the expectation and constant $B$ are removed. Then the task state $\mathbf{s}^{*}(r)$ can be selected as the minimizer of ${\hat{e}_{s}}(r)$, $\forall s\in\mathcal{S}$.

After the task state $\mathbf{s}^{*}(r)$ is selected, the mobile machine observes the exact value of random events at current task frame $r$, and then the network association decision can be made to minimize the deterministic minimization problem of (21), which can be given by

[TABLE]

Moreover, the optimal parameters of energy consumption, task frame size and state transition probability can be given by

[TABLE]

[TABLE]

[TABLE]

III-C The Online Proactive Network Association Decision

Under the guidance of optimal parameters obtained by (24)-(26), the following optimization problem can be constructed equivalently as follows

[TABLE]

where the task state is observed from network environment instead of being determined. Moreover, constraints C1 and C2 can jointly ensure that the time-averaged energy consumption of P1 is satisfied. Constraint C3 indicates that the frame fraction of state transition is equal to the optimal state transition probability. Similarly, the virtual queues can be introduced as follows

[TABLE]

[TABLE]

[TABLE]

Define the drift-plus-penalty function as

[TABLE]

where $\hat{\Theta}(r)=\{{\hat{E}}(r),{\hat{F}}(r),\hat{G}_{sm}(r),\forall s,m\in{\cal S}\}$ collects all virtual queues, $\hat{\Delta}(r)$ is conditional Lyapunov drift and $\hat{V}$ is the Lyapunov control parameter. Then the upper bound of (31) can be derived in Lemma 2.

Lemma 2

The Lyapunov drift-plus-penalty function (31) satisfies as follows:

[TABLE]

where $\hat{B}$ is a finite constant.

The proof process is omitted due to the similarity to Lemma 1. Then the proactive network association can be performed to solve the deterministic optimization problem in right-hand side of (32) at each task frame. Moreover, the detailed procedure of TOPNA to execute online proactive network association decision can be described in Algorithm 1.

IV Simulation Results

In this section, we compare the performance of TOPNA with other three network association policies, that is, “Best channel”, “Max-sojourn” and “Myopic optimal” policies indicate that the mobile machine selects MEC server within candidate association set according to the best channel quality, the maximum sojourn time and the minimum delay relative to transmission and computation, respectively.

In our simulation, the MEC servers are deployed on a regular grid network within a square area of $1000\times{\rm{1000}}\;{{\rm{m}}^{2}}$, and the coverage radius of server $Ra$ is $200\rm{m}$. The mobile machine’s trajectory is taken from real-world GeoLife dataset [16]. The transmit power of mobile machine is $p_{\textrm{tx}}=23\;\textrm{dbm}$. The computation intensity of task is ${\varepsilon}=238\;{\rm{CPU\;cycles/bit}}$. The number of past samples for random events is $W=50$. Specifically, the task data size satisfies ${\alpha}(r)\sim U(0.5,1)\;\textrm{Mbit}$, where $U(a,b)$ denotes the uniform distribution with parameters $a$ and $b$. Moreover, we assume that the average channel capacity $c_{j}$, $\forall j\in\mathcal{M}$ takes values from $[20,100]\;\textrm{Mbps}$ with evenly spaced intervals, then the channel capacity of each task frame satisfies ${c_{j}}(r)\sim U(0.5{c_{j}},1.5{c_{j}})$. Similarly, we assume that the average server’s CPU-cycle frequency ${f_{j}}$, $j\in\mathcal{M}$ takes values from $[10,50]\;\textrm{GHz}$ with evenly intervals, then the available CPU-cycle frequency for mobile machine ${f_{j}}(r)\sim U(0.5{f_{j}},1.5{f_{j}})$. Furthermore, the Lyapunov control parameters of TOPNA are set to the same value, i.e., $V=\hat{V}$. Each experiment is run in MATLAB for $3000$ task frames.

In Fig. 3, we plot the average task delay against the number of MEC servers for proposed TOPNA and other comparison schemes. We obtain that the average task delay of each scheme coincides exactly when $M=4$, this is due to no overlapping coverage of MEC servers, that is, the candidate server is unique for each task. As $M$ increases, the average task delay of proposed TOPNA algorithm gradually decreases due to the increased chance to associate the optimal server through more overlapping coverage areas, and is lower than other schemes by comprehensively considering the impacts of transmission, computation, handover and future mobility on task delay. As a result, it shows that our propose TOPNA algorithm performs more delay-efficient than other three comparison schemes, especially in dense deployment of MEC server.

Fig. 4 shows the average task delay and time-average energy consumption of proposed TOPNA method and other three schemes with increasing $V$. We find that, when control parameter $V\leq 50$, the average task delay and time-average energy consumption accordingly decrease and increase rapidly, and then tend to be stable due to the limited channel capacity and server CPU-cycle frequency when $V\geq 100$. Meanwhile, the average task delay of the proposed TOPNA satisfies the asymptotic optimality with bounded deviation and has a trade-off relationship with the time-average energy consumption. Besides, we also plot the performance of other three schemes, each of which is independent of $V$ and therefore remains unchanged. Moreover, the simulation results provide the principle that the control parameter $V$ is selected in practice, that is, under the energy consumption budget, selecting the appropriate $V$ can minimize the average task delay performance.

V Conclusion

In this paper, we propose an online proactive network association method based on event-triggered distributed computing environment, to minimize the average task delay subjected to time-averaged energy consumption for mobile machines in IoT under the assistance of mobile edge computing. We first formulate an event-triggered computing and network association model by taking into account semi-Markov task states and i.i.d. random events. Then a mobility-aware anticipatory association is developed to consider the handover caused by mobile machine’s mobility. Finally, the two-stage online proactive network association (TOPNA) decision algorithm based on the MDP and Lyapunov optimization is proposed. Simulation results demonstrate the effectiveness of our proposed algorithm. Future research directions include establishing a more general method of proactive network association for multiple machines in distributing computing and networking scenario.

Appendix A

For any positive real number $a$, $b$, $c$, the following inequality holds:

[TABLE]

According to the above property (33), and taking squares of both sides of (17) and (18), we have

[TABLE]

Similarly, we have

[TABLE]

Summing up (34) and (35), then taking expectation on both sides of and plugging into (20), we can obtain the desired result in (21). And the constant $B$ satisfies the following:

[TABLE]

where $e_{\max}$ and $T_{\max}$ represent the upper bound of energy consumption and task frame size, respectively.