Maximum Lifetime Analytics in IoT Networks

TL;DR

This paper introduces an online resource allocation algorithm for IoT networks that maximizes both network lifetime and data analytics performance, proving convergence even with time-varying conditions, and demonstrates significant lifetime improvements.

Contribution

It maps a complex joint optimization problem to a convex form and develops a provably convergent online algorithm without prior knowledge of network dynamics.

Findings

The algorithm achieves both maximum network lifetime and data analytics performance.

Network connectivity significantly impacts lifetime maximization.

Lifetime increases by approximately 50% compared to energy minimization methods.

Abstract

This paper studies the problem of allocating bandwidth and computation resources to data analytics tasks in Internet of Things (IoT) networks. IoT nodes are powered by batteries, can process (some of) the data locally, and the quality grade or performance of how data analytics tasks are carried out depends on where these are executed. The goal is to design a resource allocation algorithm that jointly maximizes the network lifetime and the performance of the data analytics tasks subject to energy constraints. This joint maximization problem is challenging with coupled resource constraints that induce non-convexity. We first show that the problem can be mapped to an equivalent convex problem, and then propose an online algorithm that provably solves the problem and does not require any a priori knowledge of the time-varying wireless link capacities and data analytics arrival process statistics. The algorithm's optimality properties are derived using an analysis which, to the best of our knowledge, proves for the first time the convergence of the dual subgradient method with time-varying sets. Our simulations seeded by real IoT device energy measurements, show that the network connectivity plays a crucial role in network lifetime maximization, that the algorithm can obtain both maximum network lifetime and maximum data analytics performance in addition to maximizing the joint objective, and that the algorithm increases the network lifetime by approximately 50% compared to an algorithm that minimizes the total energy consumption.

Figures

Equations

$$\displaystyle\bm{\lambda}=\big{(}\lambda_{i}^{(c)}:i\in\mathcal{N},c\in\mathcal{C}\big{)}.$$

$$I(i,j):=\{(a,d),(d,b):d\in {\mathcal{N}}_i\cup {\mathcal{N}}_j,a,b\in {\mathcal{N}}_d\},$$

$$\sum _{c\in \mathcal{C}}{x}_{ij}^{(c)}\le {\mu }_{ij},\sum _{c\in \mathcal{C}}{\gamma }_i^{(c)}{y}_i^{(c)}\le {\rho }_i\forall i\in \mathcal{N},(i,j)\in \mathcal{E}.$$

$$\sum _{c\in \mathcal{C}}\frac{x_{ij}^{(c)}}{{\mu }_{ij}}+\sum _{(k,m)\in I(i,j)}\sum _{c\in \mathcal{C}}\frac{x_{km}^{(c)}}{{\mu }_{km}}\le 1$$

$$\sum _{j\in {\mathcal{N}}_i}{x}_{ji}^{(c)}+{\beta }_i^{(c)}{y}_i^{(c)}+{\lambda }_i^{(c)}=\sum _{j\in {\mathcal{N}}_i}{x}_{ij}^{(c)}+y_i^{(c)}.$$

$$T_s{p}_i(\mathbf{x},\mathbf{y})\le E_i$$

$$p_i(\mathbf{x},\mathbf{y}):=\sum _{j\in {\mathcal{N}}_i}\sum _{c\in \mathcal{C}}{x}_{ij}^{(c)}{e}_{ij}^{\text{tx}}+\sum _{j\in {\mathcal{N}}_i}\sum _{c\in \mathcal{C}}{x}_{ji}^{(c)}{e}_{ji}^{\text{rx}}+\sum _{c\in \mathcal{C}}{y}_i^{(c)}{e}_i^{\text{pr}}$$

$$X$$

$$Y$$

$$\Gamma (\mathbf{\lambda }):=\{\mathbf{x}\in X,\mathbf{y}\in Y\mid \text{constraints }(\text{ref}eq:plu{s}_{f}lo{w}_{c}onstraint),(\text{ref}eq:energy-budget)\text{ are satisfied}\}$$

$$\underset{i\in \mathcal{N}}{\max }\left\{p_i({\mathbf{x}}^{\star },{\mathbf{y}}^{\star })/E_i\right\}=1/T_s^{\star },$$

$$\sum _{k=1}^K{\pi }_k\underset{i\in {\mathcal{M}}_k}{\max }\{p_i(\mathbf{x},\mathbf{y})/E_i\}.$$

$$\sum _{c\in \mathcal{C}}{x}_{ij}^{(c)}[t]\le {\mu }_{ij}[t],\sum _{c\in \mathcal{C}}{\gamma }_i^{(c)}{y}_i^{(c)}[t]\le {\rho }_i[t]$$

$$\mathbb{I}\left({\sum }_{c\in \mathcal{C}}{x}_{ij}^{(c)}[t]\right)+{\sum }_{(k,m)\in I(i,j)}\mathbb{I}\left({\sum }_{c\in \mathcal{C}}{x}_{km}^{(c)}[t]\right)\le 1,$$

$$X[t]$$

$$Y[t]$$

$$L(\mathbf{x},\mathbf{y},\mathbf{\nu })=(1-\theta )\underset{i\in \mathcal{N}}{\max }\left\{\frac{p_i(\mathbf{x},\mathbf{y})}{E_i}\right\}+\theta \sum _{i\in \mathcal{N},c\in \mathcal{C}}-{\omega }_i^{(c)}(y_i^{(c)})$$

$$+\sum _{i\in \mathcal{N}}\sum _{c\in \mathcal{C}}{\nu }_i^{(c)}\left(\sum _{j\in {\mathcal{N}}_i}(x_{ji}^{(c)}-x_{ij}^{(c)})+({\beta }_i^{(c)}-1)y_i^{(c)}+{\lambda }_i^{(c)}\right)$$

$$h(\mathbf{\nu }):=\underset{\mathbf{x}\in X,\mathbf{y}\in Y}{\min }L(\mathbf{x},\mathbf{y},\mathbf{\nu }),$$

$${\nu }_i^{(c)}[t+1]=$$

$$\displaystyle\textstyle+(\beta_{i}^{(c)}-1)y_{i}^{(c)}[t]+\lambda_{i}^{(c)}\Big{)}\qquad\forall c\in\mathcal{C},i\in\mathcal{N},$$

$$(\mathbf{x}[t],\mathbf{y}[t])\in \underset{\mathbf{x}\in X,\mathbf{y}\in Y}{\mathrm{argmin}}L(\mathbf{x},\mathbf{y},\mathbf{\nu }),$$

$$X:=\underset{T\to \infty }{\lim }\frac{1}{T}\underset{t=1}{\overset{T}{⨁}}X[t],Y:=\underset{T\to \infty }{\lim }\frac{1}{T}\underset{t=1}{\overset{T}{⨁}}Y[t]$$

$$x^{\star }=\underset{T\to \infty }{\lim }\frac{1}{T}\sum _{t=1}^T{x}^{\star }[t],y^{\star }=\underset{T\to \infty }{\lim }\frac{1}{T}\sum _{t=1}^T{y}^{\star }[t]$$

$$\text{(i)}\frac{1}{T}\sum _{t=1}^T\mathbf{E}\left(f(x[t],y[t])-f(x^{\star }[t],y^{\star }[t])\right)\le \alpha {ϵ}_1+{ϵ}_2,$$

$$\text{(ii)}\underset{T\to \infty }{\lim }\mathbf{E}\left(({\bar{x}}_{ji}^{(c)}-{\bar{x}}_{ij}^{(c)})-({\beta }_i^{(c)}-1){\bar{y}}_i^{(c)}+{\lambda }_i^{(c)}[t]\right)=0,$$

$$\displaystyle\big{(}\bm{x}[t],\bm{y}[t]\big{)}\in\!\!\!\!\operatorname*{\arg\min}_{\begin{subarray}{c}\bm{x}\in X[t],\bm{y}\in Y[t]\end{subarray}}\tilde{L}(\bm{x},\bm{y},\alpha\bm{\nu}[t])$$

$$\frac{1}{T}{\sum }_{t=1}^T(f(x[t],y[t])-f(x^{\star }[t],y^{\star }[t]))$$

$$\le -\frac{1}{T}{\sum }_{t=1}^T(\nu [t]{)}^{\top }(g(x[t],y[t])+\lambda [t])$$

$$+\frac{1}{T}{\sum }_{t=1}^T(\nu [t]{)}^{\top }(g(x^{\star }[t],y^{\star }[t])+\lambda [t])+{ϵ}_2$$

Full text

I Introduction

We consider an Internet of Things (IoT) network where a set of nodes collect measurements that have to be later on analyzed by a data analytics or machine learning (ML) algorithm. For example, an algorithm for classifying, filtering or summarizing data. This class of services is one of the most important envisioned applications of the emerging IoT networks [1] and poses many technical challenges [2]; especially, when IoT networks operate subject to bandwidth, processing, and energy constraints.

Unlike previous generations of sensor networks, it is expected that IoT applications collect data at an unprecedented rate and that only a fraction of these will be non-ephemeral [3, 4]. Hence, the usual approach of transferring all the data to an IoT node gateway for processing, even if possible, may consume network resources unnecessarily. A promising solution to overcome this issue is to leverage the processing capacity of the IoT nodes and execute (some of) the data analytics tasks at the edge. That is, by carrying out some of the processing in situ it is possible to reduce the amount of data that needs to be transmitted over the network. However, executing data analytics at the IoT nodes has some costs. The two most significant ones are perhaps that processing is expensive in terms of energy, and that the grade or performance of how a task is carried out depends on the algorithms that IoT nodes are able to run. For example, nodes with limited memory can only classify data with low complexity models, which makes their predictions less accurate in general.

In this paper, we introduce a resource allocation framework that allows us to design online execution and routing policies for data analytics tasks in IoT networks. That is, decide whether a data analytics task should be processed (i) locally, (ii) at the gateway, or (iii) elsewhere in the IoT network (e.g., at a neighboring IoT node) depending on the available network resources (bandwidth, processing capacity and energy). An important feature of the framework is that it also allows us to maximize a combined criterion of network lifetime111Time the network can operate without any node running out of battery. and data analytics performance. The first objective is important because we would like the network to operate for as long as possible, and the second because the network has to fulfill its purpose besides “staying alive”. We call this problem Maximum Lifetime IoT Analytics (MLIA).

The problem of maximizing the time a network can operate has been addressed before in sensor networks (see Section II). However, unlike sensor networks, IoT encompasses more sophisticated scenarios where a variety of heterogeneous devices and applications coexist [2], and brings data analytics processing into play. The latter adds, technically, a new dimension to the existing routing algorithms (e.g., [5, 6, 7, 8]) and raises technical challenges that cannot be addressed with previous solutions directly. In particular, to capture in-network processing we need to use a network model with “gains” (Section III-B), transform a non-convex problem into an equivalent convex one (Lemma 1), and develop a new online algorithm (Theorem 1) that can handle non-linearities in the utility as well as randomness in the actions (i.e., the routing and processing decisions that can be made in each time slot). Furthermore, the fact that nodes can only operate during a limited time span222Due to energy constraints. adds the difficulty of selecting the algorithm parameters so as to obtain the desired performance (see Section VI-B2). To this end, the main contributions of the paper are:

• (i)

MLIA problem: we introduce the problem of jointly maximizing the lifetime of an IoT network and the performance of how data analytics tasks are carried out. This is an open problem arising in many IoT applications.

• (ii)

Problem model: we formulate the MLIA problem as a convex optimization program. The model captures aspects such as the coexistence of different types of data analytics tasks, that data analytics may be carried out by different IoT nodes, and that their computation cost may vary across nodes, among others.

• (iii)

Online algorithm: we propose an online algorithm that solves the underlying convex problem and has non-asymptotic convergence guarantees. The algorithm determines the joint routing and processing policy for the IoT nodes in a myopic manner by only looking at the system’s current state, i.e., it does not require statistical knowledge of the underlying random processes such as the time-varying link capacities. Also, and to the best of our knowledge, this is the first paper that presents the convergence of the dual subgradient method with time-varying sets, which is another contribution.

• (iv)

Performance evaluation: we perform a set of extensive experiments to (i) evaluate the performance of the proposed solution and to (ii) understand how this is affected by the network connectivity and system parameters. We also compare our algorithm to two benchmark policies, and show that our approach can increase the network lifetime by approximately $50\%$ compared to an algorithm that minimizes the total energy consumption.

The rest of the paper is organized as follows. Section II presents the related work. In Section III, we introduce the system model, the problem, and the arising trade-offs. In Section IV, we formulate the MLIA problem as a convex program, and in Section V, we present the online algorithm. Section VI contains the numerical experiments and discussion.

II Related Work

The problem of deciding how to transmit and process data to prolong the network lifetime has been studied before in Wireless Sensor Networks (WSNs). For instance, the work in [9] proposes load-balancing techniques to spread the energy consumption across nodes. In [10], the authors study the problem of designing a medium access protocol that takes into account the channel state information and the available energy. And in [11], it is shown that preprocessing data at the sensor nodes can help to reduce the network load and so the energy required to transmit data to the fusion center (i.e., the gateway). From a problem formulation perspective, our approach differs from previous works in the literature of WSNs and IoT because we consider both the energy spent in routing and processing the data, and the performance of the analytics. The last point is crucial in heterogenous IoT networks where the ability of nodes to run algorithms depends on their hardware.

The maximum lifetime objective has been extensively studied for routing in multi-hop sensor networks. The seminal work in [6] considered a static maximum lifetime routing problem and formulated it as a linear program. This work was extended in a sequel of papers [12, 5] to different types of wireless networks, and a flow augmentation algorithm was proposed to support fixed and arbitrary generation rates. The approach in [6] has also been adopted by other authors (see survey [13]); for example, [14] combines network lifetime with congestion control, and [7] proposes an algorithm to solve the maximum lifetime problem in a distributed fashion. Finally, [15] proposed a static optimization solution for maximizing analytics performance in IoT networks with average power constraints, which is different than the lifetime criterion. Our work differs technically from all previous work on maximum lifetime routing in sensor networks because (i) it considers IoT sensing nodes that are heterogeneous and can perform data analytics computations in addition to routing data; (ii) it uses an objective that incorporates performance in addition to lifetime; and (iii) the proposed dynamic algorithm solves, provably, the underlying convex problem without knowledge of the arrivals or channel statistics. Regarding the algorithm, we use time-varying sets to handle the instantaneous routing and processing constraints, which is in marked contrast to stochastic Lagrange dual approaches where the stochasticity does not affect the decision variables.

Finally, we note that data analytics optimization with routing costs has been considered in the cloud context [16, 17]. However, there the costs are not related to energy expenditure, neither the resource allocation decisions affect the time the network will be able to operate. Furthermore, in IoT, we have the additional inherent difficulties of routing data over wireless networks which do not appear, of course, in the cloud.

III System Model and Problem Statement

III-A System model

III-A1 Network

We model an IoT network as a directed graph $\mathcal{G}=(\mathcal{N},\mathcal{E})$ consisting of $n=|\mathcal{N}|$ nodes and $l=|\mathcal{E}|$ links. We use $\mathcal{C}$ to denote the set of applications in the network. For each application $c\in\mathcal{C}$, the nodes collect and send data to a gateway node—possibly over a multi-hop path. Parameter $\lambda_{i}^{(c)}$, $i\in\mathcal{N}$, $c\in\mathcal{C}$ indicates the rate at which node $i$ collects data of application $c$. These are exogenously given and injected into the network directly. The arrival rate matrix is given by

[TABLE]

Each link $(i,j)\in\mathcal{E}$ has an average capacity of $\mu_{ij}$ $\mathrm{bits}\mathrm{/}\mathrm{s}$; we collect these in matrix $\bm{\mu}=(\mu_{ij}:i,j\in\mathcal{N})$. Links $(i,j)$ and $(j,i)$ can have different capacities. The transmission over link $(i,j)$ induces an energy consumption $e_{ij}^{\text{tx}}$ for the sender node $i$, and $e_{ij}^{\text{rx}}$ for the receiver node $j$. Both are measured in $\mathrm{J}\mathrm{/}\mathrm{bit}$. We model the wireless interference using the protocol interference model [18], according to which a transmission over link $(i,j)\in\mathcal{E}$ is successful if and only if all nodes in range with $i$ or $j$ are idle. This requirement is based on the CSMA/CA protocol adopted by IEEE 802.11 standards.333In detail, this interference model complies with the communication sequence RTS-CTS-Data-ACK, where each sender is also a receiver of the ACK packets; which is the strictest model. The set of interfering nodes can be reduced if the ACK operation is not used. The set of interfering links for each link $(i,j)\in\mathcal{E}$ is defined as:

[TABLE]

where $\mathcal{N}_{i}:=\{j\in\mathcal{N}:(i,j)\in\mathcal{E}\}$ is the set that contains the neighbors of a node $i\in\mathcal{N}$. Hence, when a link $(i,j)$ is active none of the links in $I(i,j)$ can be used.

III-A2 Nodes

IoT nodes may be heterogenous in terms of hardware. We use $\rho_{i}$ to denote the processing capacity in $\mathrm{FLOPS}$ of a node $i\in\mathcal{N}$, and $\gamma_{i}^{(c)}$ to denote the processing requirements (in $\mathrm{FLOPS}\mathrm{/}\mathrm{bit}$) of an application in each of the nodes. The processing in each node may reduce the volume of each flow by a factor of $0\leq\beta_{i}^{(c)}\leq 1$. For example, in an object recognition application the flow volume reduction is large since an image gets reduced to a collection of bounding boxes and tags [19], i.e., to few bytes. Of course, this flow reduction factor may depend on where a data analytics task is carried out since IoT nodes may run different algorithms. We collect the flow reduction and processing requirements in matrices $\bm{\beta}=(\beta_{i}^{(c)}:i\in\mathcal{N},c\in\mathcal{C})$ and $\bm{\gamma}=(\gamma^{(c)}_{i}:i\in\mathcal{N},c\in\cal C)$.

Each IoT node $i\in\mathcal{N}$ has an energy budget of $E_{i}$ Joules that can spend transmitting, receiving, and processing data. When a node has used its energy budget, it dies meaning that it cannot transmit or process more data. We assume that the network gateway does not have energy constraints.

III-B Decision variables and constraints

IoT nodes can make two types of decisions: process and forward data. Variable $x_{ij}^{(c)}\geq 0$ indicates the rate in $\mathrm{bits}\mathrm{/}\mathrm{s}$ at which application $c$ is transmitted over link $(i,j)$. Similarly, variable $y_{i}^{(c)}$ indicates the rate in $\mathrm{bits}\mathrm{/}\mathrm{s}$ that node $i$ processes data of application $c$. We collect the decision variables in matrices $\bm{x}=(x_{ij}^{(c)}:\,(i,j)\in\mathcal{E},\,c\in\mathcal{C})$ and $\bm{y}=(y_{i}^{(c)}:\,i\in\mathcal{N},\,c\in\mathcal{C})$. The transmission and processing rates must satisfy the link and node capacity constraints:

[TABLE]

The interference constraints affect how the links in $\mathcal{E}$ can be activated and consequenctly the total amount of data that can be transferred over the network; see, for instance, Lemma 1 in [20]. These are formally given by:

[TABLE]

where $I(i,j)$ is defined in (2).

Unlike classic max-flow type models [21], in processing-capable networks the amount of flow that arrives at a node may not be the same as the amount of flow that departs. Now, the flow conservation constraints are given by

[TABLE]

Equation (5) says that the data received from the other nodes in the network, plus the data received after the local processing, plus the exogenous arrivals must be equal to the traffic sent to other nodes in the network and for processing. Figure 1 shows an example of a network with five IoT nodes and a gateway.

Finally, the decisions $\bm{x}$ and $\bm{y}$ need to respect the energy budget of each node. These must satisfy:

[TABLE]

where

[TABLE]

and $T_{s}$ is the network life time:

Definition 1** (Network lifetime).**

Time span where all the IoT nodes are alive, i.e., none of them runs out of energy.

Hence, (6) must be satisfied by all the nodes in the network. We later extend this definition to scenarios where a subset of nodes are allowed to die before the network lifetime expires.

III-C Utilities

The execution of analytics at the IoT nodes and the gateway results in different performance. The latter can be related, for instance, to the precision or confidence that an image is classified using a machine learning algorithm. Formally, we define function $\omega_{i}^{(c)}:{\mathbf{R}}\to{\mathbf{R}}$, $i\in\mathcal{N},c\in\mathcal{C}$ to capture the network benefit of processing an application at a node. We assume $\omega_{i}^{(c)}$ is concave and non-decreasing, and that IoT nodes may have different performance for processing the same task. This performance diversity may arise due to hardware or software differences among the IoT nodes. See [19, Figure 1] for an example of how different object detection algorithms have different precisions.

III-D Problem statement

The IoT nodes collect data that must be processed locally, at another IoT node, or at the network gateway. Ideally, we would prefer to process all the data in the gateway since this has usually an “unlimited” energy budget and better hardware that allows it to run more sophisticated algorithms. However, that might not be always possible for several reasons. First, the amount of bandwidth available between the nodes and the gateway might not be enough to transport all the data. Second, nodes are subject to energy constraints, which in turn limit the total amount of data that can be transferred. And third, the central node may not be able to cope with the processing of all the data from all the IoT nodes.

Our goal is to design a routing and processing policy ($\bm{x},\bm{y}$) that maximizes (i) the network lifetime and (ii) the analytics performance. The lifetime criterion is crucial because when nodes run out of battery, the network operation is disrupted e.g., nodes stop collecting information, monitoring an area, among others. On the other hand, the analytics performance metric is important because our goal is not just to keep the nodes in the network alive but also to maximize the service performance. Sometimes these two objectives may not be conflicting since processing data in the network reduces the network congestion and so the subsequent routing energy cost. However, in-network processing incurs an energy cost as well as analytics performance degradation since IoT nodes may run “lighter” algorithms that fit their hardware specifications. The overall balance depends on many parameters, such as the flow reduction due to processing, the energy cost of these tasks, and the network properties.

In sum, we would like to find a policy $(\bm{x},\bm{y})$ that maximizes a combined criterion of network lifetime and data analytics performance depending on (i) the capacity of network links; (ii) the nodes’ processing capacity; (iii) the applications data rates; (iv) the data analytics performance “quality”; and (v) the nodes’ energy budget. Next, we present the mathematical formulation of the problem and introduce an algorithm that is amenable to implementation in stochastic environments.

IV Maximum Network Lifetime and Analytics

The IoT operation constraints determine the transmission and processing policies that are implementable, and subsequently, the set of supportable data rates. First, we define sets:

[TABLE]

These sets are bounded polytopes (so convex sets [22, Section 2.2.4]) because they are the intersection of inequality constraints and all links and nodes have bounded capacity. Using these sets, we can define the capacity region of the IoT system.

Definition 2** (IoT network capacity region).**

The IoT network capacity region is the set

[TABLE]

where $\bm{\lambda}$ is given in (1).

We will always assume that $\Gamma(\bm{\lambda})$ is non-empty. We are now in position to formulate our optimization problem.

IV-A Maximum lifetime data analytics problem

Recall that $T_{s}$ is the network lifetime. The optimization problem is given by

[TABLE]

Parameter $\eta\in[0,1]$ is used to balance how much we prioritize the network lifetime over the analytics performance metric. If $\eta=0$, then (10) aims only to maximize the network lifetime; if $\eta=1$, the optimization only takes into account the data analytics performance; and for any value of $\eta\in(0,1)$, it balances the two terms in the objective.

Problem (10) is non-convex because the inequality constraint (6) involves the product of variables $\bm{x}$ and $T_{s}$.444The product of two variables is generally not convex. Moreover, variables $(\bm{x},\bm{y})$ and $T_{s}$ are not independent since $(\bm{x},\bm{y})$ affect the energy consumption and so indirectly the time the network will be able to operate. Nonetheless, and as we show in the following lemma, it is possible to transform the non-convex problem (10) into an “equivalent” convex problem.

Lemma 1**.**

The optimization problem

[TABLE]

is convex and “equivalent” to the non-convex problem in (10). It allows us to balance smoothly between maximum network lifetime and maximum data analytics performance.

Proof:

Let $(T_{s}^{\star},\bm{x}^{\star},\bm{y}^{\star})$ be a solution to (10). By Weierstrass’ theorem [23, Proposition 2.1.1; condition 1], one can show that a solution always exists and is finite.555Note that by construction $X$ and $Y$ are bounded sets. Variable $T_{s}$ belongs also to a bounded set since it is nonnegative and nodes have a finite energy. The key part of the proof relies on showing that one of the energy constraints must be tight at the optimum. We proceed to show this by contradiction. Suppose there exists a $T^{\prime}>T^{\star}_{s}$ such that $T^{\star}_{s}p_{i}(\bm{x},\bm{y})<T^{\prime}p_{i}(\bm{x}^{\star},\bm{y}^{\star})\leq E_{i}$ for all $i\in\mathcal{N}$. Then, we must also have that $(1-\eta)T^{\star}_{s}<(1-\eta)T^{\prime}$ (i.e., the objective value increases); however, this is not possible since by assumption $T_{s}^{\star}$ is an optimal value. Hence, we have that at least one constraint must be tight at the optimum, i.e., $T^{\star}_{s}p_{i}(\bm{x}^{\star},\bm{y}^{\star})=E_{i}$. Next, rearrange terms in the last equation and rewrite the energy constraints as

[TABLE]

where the $\max$ follows because the constraint must be satisfied by all the nodes. From (14), we can see that maximizing $T_{s}^{\star}$ is equivalent to minimizing the LHS of (14)—which is a convex function since $p_{i}(\bm{x},\bm{y})$ is linear [22, pp. 72] and $E_{i}$ is a constant. Finally, considering that $\sum_{i\in\mathcal{N},c\in\mathcal{C}}-\omega_{i}^{(c)}(y_{i}^{(c)})$ is convex, we can use scalarization with $\theta\in[0,1]$ to obtain a convex problem. ∎

It is important to emphasize that problems (10) and (13) are equivalent but not the same. Namely, if $\theta=0$, then the optimization maximizes the network life time; if $\theta=1$, it only takes into account the analytics performance; and if $\theta\in(0,1)$ it balances the two objectives smoothly. However, the solutions to problems (10) and (13) do not need to be the same for $\theta=\eta$ when $\theta\in(0,1)$ since the network lifetime term in (10) is linear, whereas in (13), the term must be regarded as $1/T_{s}$. As we will show in detail in Section VI, parameter $\theta$ has a huge impact on the system’s performance.

IV-B Generalized maximum network lifetime problem

Instead of considering the lifetime of individual nodes, we can consider the lifetime of groups of nodes $\mathcal{M}_{k}\subset\mathcal{N}$, $k\in\{1,2,\ldots,K\}$. For instance, the type of data collected by a subset of nodes may be more important to the global system objective than the data collected by another subset of nodes. To consider groups of nodes in the optimization problem, we can replace the network lifetime term in the objective of problem (13) with

[TABLE]

Parameter $\pi_{k}\geq 0$ is used to emphasize how much we would like a subset of nodes to “remain” alive with respect to another subset. Note that the formulation in (13) is a special case where $|\mathcal{M}_{k}|=1$ for all $k\in\{1,\dots,K\}$ and $\cup_{k=1}^{K}\mathcal{M}_{k}=\mathcal{N}$. Hereafter, and to streamline exposition, we will use the formulation in (13) but our results apply to more general cases directly.

IV-C Practical limitations

The resulting optimization problem is readily solvable by convex solvers such as SCS [24]. However, for this, one needs to know all the system parameters, which is rarely the case in practical scenarios. For example, the average capacity of a link connecting two IoT nodes is usually not known. Furthermore, there are generally instantaneous constraints, such as the level of noise in the system or interference, which affect the decisions that the IoT nodes can make. In the next section, we present a dynamic algorithm that learns and adapts to the instantaneous network/system characteristics.

V Dynamic MLIA Algorithm

In this section, we present a dynamic algorithm that solves (13) and does not require previous knowledge on (i) the average arrival rate of the data analytics; (ii) the capacity of the network links $\bm{\mu}$ in each time instance; and (iii) the average performance or reward obtained from carrying out data analytics at the IoT nodes.

V-A Dynamic problem formulation

We divide the time in slots $t\in{\mathbf{N}}$ of normalized duration and parameterize the variables in the static model (13) with $[t]$ to indicate their value in a time slot. For instance, $\lambda_{i}^{(c)}[t]$ indicates the new arrivals of application $c\in\mathcal{C}$ at node $i$ and time $t\in{\mathbf{N}}$, and $x^{(c)}_{ij}[t]$ is the amount of commodity transmitted over link $(i,j)\in\mathcal{E}$. We also need to capture the network constraints of the dynamic problem. The instantaneous link capacity constraints are

[TABLE]

$\forall(i,j)\in\mathcal{E},i\in\mathcal{N}$. The (binary) interference constraints are

[TABLE]

for all $(i,j)\in\mathcal{E}$ where $\mathbb{I}$ is the indicator function, i.e., $\mathbb{I}(x)=1$ if $x>0$ and $\mathbb{I}(x)=0$ otherwise. Hence, the binary interference constraints only allow one node to transmit in the interference range regardless of the transmission rate. We are now in position to define sets

[TABLE]

which contain the admissible routing/processing policies that can be implemented in the system in each time slot. Note that $X[t]$ may not be convex since (16) is not convex.

The operation of the network consists of selecting actions (or values) from the instantaneous action sets $X[t]$ and $Y[t]$ while taking into account the properties of the underlying convex problem. We explain next how this can be achieved.

V-B Algorithm overview

The key idea for solving (13) in the dynamic setting is to relax the flow conservation constraints and formulate the Lagrange dual problem. The Lagrangian is

[TABLE]

where $\nu_{i}^{(c)}\in{\mathbf{R}}$ is the Lagrange multiplier associated to each of the flow conservation constraints, and we define $\bm{\nu}=(\nu^{(c)}:i\in\mathcal{N},c\in\mathcal{C})$. The Lagrange dual function is defined as

[TABLE]

and recall that it is concave [22, Section 5.2]. Hence, it can be maximized with the subgradient method applied to the Lagrange dual problem.666We use subgradient instead of gradient because the Lagrange dual function may not be differentiable. Specifically, with update:

[TABLE]

where the term in parenthesis is a subgradient of $h$ at $\bm{\nu}[t]$,

[TABLE]

and $\alpha>0$ the step size or parameter that controls the accuracy of solution in the optimization. Recall that the solution of the primal and dual problem coincides when strong duality holds [22, Section 5.2.3]; which is the case in our problem since the constraints in (13) are linear and the network capacity or feasible set $\Gamma(\bm{\lambda})$ is non-empty by assumption.

Next, we proceed to explain how to include constraints (15)-(16) so that our dynamic algorithm can be implemented in a real system. We do this incrementally from the static problem.

V-B1 Unknown arrivals

Suppose now that $X[t]=X$ and $Y[t]=Y$ for all $t\in{\mathbf{N}}$ (the sets of actions do not change over time). We want to relax the fact that the data analytics arrival $\bm{\lambda}$ is not known a priori. For this, we need to solve the Lagrange dual problem (i.e., maximize $h(\bm{\nu})$) with the subgradient method replacing $\lambda_{i}^{(c)}$ with the random variable $\lambda_{i}^{(c)}[t]$ in the dual update (18). This change amounts to making the subgradient of $h(\bm{\nu})$ stochastic, or equivalently, to using the stochastic dual subgradient method to solve the underlying convex problem (13). For the algorithm to converge we need to make the following assumption.

Assumption 1**.**

$\{\lambda_{i}^{(c)}[t]\}$* is an i.i.d. process with expected value $\lambda_{i}^{(c)}$ for all $i\in\mathcal{N}$, $c\in\mathcal{C}$, $t\in{\mathbf{N}}$ and $\lambda^{(c)}_{i}[t]$ is uniformly bounded for all $t\in{\mathbf{N}}$.*

V-B2 Time-varying sets

We can incorporate time-varying sets in our algorithm by replacing $X$ and $Y$ with $X[t]$ and $Y[t]$ in update (19). In order to solve the underlying problem (13), we need to make the following assumption, which can be regarded as if we used stochastic subgradients in update (18).

Assumption 2**.**

${\mathbf{E}}(X[t])=X$* and ${\mathbf{E}}(Y[t])=Y$ for all $t\in{\mathbf{N}}$ and $X[t]$ and $Y[t]$ are bounded sets. Here, the expectations are defined using (Minkowski) set addition [25, pp. 32].*

V-B3 Noisy utility function

We may not have access to the utility function that measures the exact reward derived from processing analytics locally. To capture this, we define $\tilde{\omega}^{(c)}_{i}:{\mathbf{R}}\to{\mathbf{R}},i\in\mathcal{N},c\in\mathcal{C}$ to be an estimate of utility function $w_{i}^{(c)}$. Also, let $\tilde{L}(\bm{x},\bm{y},\bm{\nu})$ be the Lagrangian where $\tilde{\omega}^{(c)}_{i}$ is used instead of $\omega^{(c)}_{i}$, and replace the Lagrangian in (19) with $\tilde{L}(\bm{x},\bm{y},\bm{\nu})$. Technically, minimizing $\tilde{L}(\bm{x},\bm{y},\bm{\nu})$ instead of $L(\bm{x},\bm{y},\bm{\nu})$ corresponds to computing (sub)gradients of the Lagrange dual function approximately, i.e., having $\epsilon$-subgradient [26, pp. 625]. The convergence depends on the mild assumption that the errors are bounded.

Assumption 3**.**

The maximum error between $\omega^{(c)}_{i}$ and $\tilde{\omega}^{(c)}_{i}$ is bounded, i.e., $\max_{y\leq\rho_{i}}|\omega^{(c)}_{i}(y)-\tilde{\omega}^{(c)}_{i}(y)|:=\xi_{i}^{(c)}<\infty$.

V-C Algorithm & convergence

Algorithm 1 consists of three steps. The first one is to obtain the network state or set of possible actions, i.e., $X[t]$ and $Y[t]$. We assume these are obtained by the IoT nodes and that the information is transmitted to the network gateway where the routing decisions are made. The second step is to minimize the Lagrangian using sets $X[t]$ and $Y[t]$. The complexity of this step depends on the number of elements in the sets. When these are discrete and contain few elements, the minimization can be carried out by exhaustive search, and if $X[t]$ and $Y[t]$ are convex by standard convex optimization techniques [22, 23]. The third and final step is to update the dual variables, i.e., carry out the (stochastic) subgradient update with $\epsilon$-subgradients. Parameter $\alpha>0$ is the step size that controls the accuracy/speed tradeoff of the algorithm [23].

We establish the convergence of the algorithm with respect to an optimal policy implemented in a random fashion. We need to make the following definitions.

Definition 3** (Average policy space).**

[TABLE]

where $\oplus$ denotes the (Minkowski) set addition [25, pp. 32].

Definition 4** (Optimal policy).**

An optimal policy is a pair $(x^{\star},y^{\star})\in(X,Y)$ that solves (13). By construction we have

[TABLE]

where $x^{\star}[t]\in X[t]$ and $y^{\star}[t]\in Y[t]$ for all $t\in{\mathbf{N}}$.

Theorem 1**.**

Let $f$ be the objective function in the optimization problem (13). Suppose Assumptions 1, 2 and 3 are satisfied. Algorithm 1 ensures that

[TABLE]

$\forall i\in\mathcal{N}$, $c\in\mathcal{C}$ where $\bm{\bar{x}}:=\frac{1}{T}\sum_{t=1}^{T}{\bar{x}}[t]$, $\bm{\bar{y}}=\frac{1}{T}\sum_{t=1}^{T}\bm{y}[t]$; $\epsilon_{1}$ is a bounded constant related to the variance of the arrival process and sets $X[t]$ and $Y[t]$, and $\epsilon_{2}$ a constant such that $|\xi_{i}^{(c)}|_{2}\leq\epsilon_{2}$.

Theorem 1 establishes an upper bound on the objective function of our policy $(\bm{x}[t],\bm{y}[t])$ compared to an optimal policy $(\bm{x}^{\star},\bm{y}^{\star})$ implemented in a “randomized” fashion—that is, the policy that we would implement if we knew in advance all the parameters in the system. Note that the bound on the optimality gap depends on $\epsilon_{1}$ and $\epsilon_{2}$. The first term is related to the variation of the arrivals and link capacities, and the second on how good the estimate of the reward functions are. Importantly, note that the first term depends on the step size $\alpha$, which means that we can make it arbitrarily small. Theorem 1 establishes also that the flow conservation constraints are satisfied on expectation as $T\to\infty$, which means that we will recover, asymptotically, a policy in the network capacity region $\Gamma(\bm{\lambda})$.

V-D Practical aspects

V-D1 Non-asymptotic analysis

Differently from previous stochastic network optimization analyses, e.g., [27, 28], our results do not compare the performance of the average policy. Furthermore, and unlike these previous works that give optimality bounds only asymptotically (i.e., as $T\to\infty$), here we provide guarantees on how the policy performs on expectation per time slot. This is very important for this problem because IoT nodes have a finite energy budget that may not allow them to reach a “steady” state—this will be illustrated in the experiments in Section VI-B2.

V-D2 Distributed execution

MLIA describes a centralized algorithm where the IoT gateway collects the nodes’ parameters to solve (20). However, the only necessary central calculation is that for devising an eligible link activation schedule (based on the interference constraints). If such a schedule is already given or if the links are orthogonal (or, if any other interference control scheme is used) then each node can independently optimize its actions. Namely, each node $i$ can calculate separately the quantity $\partial\tilde{L}(\bm{x},\bm{y},\bm{\nu})/\partial\bm{x}_{ij}$, $\forall j\in\mathcal{N}_{i}$, and similarly for $y_{ij}$ variables; and then each pair of 1-hop neighbors exchange the respective dual variables. This is a standard approach for enabling a decentralized implementation of such protocols, see [25] for a survey, and applies directly to MLIA.

VI Experiments and Evaluation

We evaluate the performance of our approach numerically using real hardware and application parameters. We investigate three points. First, how parameter $\theta$ and the network connectivity affect the network lifetime and the analytics performance in the static or offline problem (13). Second, the convergence of the dynamic algorithm to the solution of the static problem depending on $\alpha$. And third, how the proposed algorithm compares to two benchmark algorithms. We fix the network size in the experiments to $20$ nodes (including the gateway), but similar results are obtained with different network sizes.

VI-A Experiments setup

VI-A1 Network

In all simulations, we use random geometric graphs (RGG) to model an IoT network—this is common practice in wireless networks [29]. Recall that in RGGs nodes are placed uniformly at random on an area of $1\times 1$ normalized units, and that their connectivity depends on the radius or distance each node covers. For simplicity, we assume that all links have an average capacity of $24$ $\mathrm{Mbits}$.

VI-A2 Nodes

The IoT nodes are Raspberry Pi(es) 3B equipped with an ARMv7 CPU, 1 $\mathrm{GB}$ RAM and a 802.11.b/g/n network card. According to our measurements, the power required to transmit and receive data is $0.4$ $\mathrm{W}$ (with small variations depending on the channel quality), and $2.1$ $\mathrm{W}$ for processing at full power. We assume that the energy spent to collect data and in idle mode is negligible.777That is, IoT nodes can switch to low-power consumption mode. All IoT nodes have batteries with a capacity of $2500$ $\mathrm{J}$;888The batteries have a small capacity to keep simulations short. An IoT node can easily be equipped with a battery that has one hundred times more energy. the gateway is connected to a constant energy source and so does not have energy constraints.

VI-A3 Application

This consists of analyzing video streams of rate $1$ $\mathrm{frame}\mathrm{/}\mathrm{s}$ with the object detection algorithm YOLO [19]. Frames have size $0.5$ $\mathrm{MB}$ and the outcome of the processing (a collection of bounding boxes and tags that indicate where the objects are in a frame; see the video in [30] for an example) can be represented with at most $0.5$ $\mathrm{kB}$. Hence, $\beta=0.5$\mathrm{MB}$/0.5$\mathrm{kB}$=10^{-3}$. We consider that an IoT node is a “source” node (receives a video stream) with probability $1/2$. According to our measurements, the IoT nodes and gateway (a desktop with a GPU) can process a frame in $3$ $\mathrm{s}$ and $100$ $\mathrm{ms}$ respectively. The analytics performance metric is given by the mean Average Precision (mAP [31]) that YOLO can detect objects correctly in a frame This is $33.1$ (YOLOv3-tiny) for an IoT node and $57.9$ (YOLOv3-608) for the gateway [30].

VI-B Evaluation

VI-B1 Sensitivity analysis

Figure 2 shows the network lifetime (left) and data analytics performance (right) for a range of values $\theta$ and nodes’ radius. Recall that the radius affects the structure of the graph (e.g., number of links, shortest path to the gateway). The results are the average of 50 different networks generated as described in the previous section. Observe from the figure that independently of the radius, the network lifetime increases monotonically as $\theta\to 0$. Nonetheless, the radius plays an important role: when the network connectivity is high,999Nodes are on average less than two hops away from the gateway. the best strategy is to offload all the processing to the gateway.101010That is because processing analytics at the nodes locally is more expensive than transmitting them, and the processing reward at the gateway is larger than the reward obtained as a result of processing the analytics at the nodes. Otherwise, the solution balances between local and gateway processing. Also, observe that in this specific example selecting $\theta=1$ is generally not the “best” choice. When the radius is equal to $0.25$, if we change from $\theta=1$ to $\theta=10^{-8}$ we can increase the network lifetime by $577\%$ at the expense of just $-21\%$ in the data analytics performance (see Figure 2). Similarly, when the radius is equal to $0.45$ (the network is dense), by changing $\theta$ from $1$ to $10^{-8}$ the network lifetime increases by $707\%$ without affecting the analytics ($0\%$ change). Conclusion: the data analytics performance and network lifetime terms in the objective are very sensitive to parameter $\theta$. Also, these may not necessarily conflicting: it is possible to obtain both, good analytics performance and network lifetime, by setting $\theta$ properly.

VI-B2 Performance of the dynamic algorithm

We now investigate how Algorithm 1 solves the static problem for a network with radius equal to $0.25$. Parameter $\theta$ is fixed to $10^{-5}$. The video frames arrive in the IoT nodes following a Poisson process. To capture the errors on the prediction function, we add noise of $20\%$. We run Algorithm 1 with $\alpha\in\{10^{-2},10^{-3},10^{-4}\}$ and show the results in Figure 3. The results are the average of 50 samples. On the left, we have the normalized optimality gap111111The optimality gap divided by $\theta$. per iteration, and on the right, the system’s congestion or sum of all the Lagrange multipliers $\nu^{(c)}_{i}$. The gray area indicates the lifetime of the system computed in the static problem.

Observe that the plots have different lengths (i.e., different network lifetimes) and convergence behavior. Specifically, with $\alpha=10^{-2}$, the transient phase is short,121212Time required to converge to a ball or steady value around the optimum. the optimality gap is large, and the network lifetime shorter than in the static problem. With $\alpha=10^{-3}$, the transient phase is slightly longer, but we obtain instead a much smaller optimality gap, and the network lifetime matches nearly the one of the static problem. Finally, with $\alpha=10^{-4}$, the transient phase is so long that the optimality gap is always negative (i.e., better than the static optimum) and the network lifetime longer than in the static problem. However, note that this is possible because congestion keeps building up in the system, i.e., data analytics get accumulated in the nodes without being processed neither transmitted. Conclusion: parameter $\alpha$ controls not only the optimality gap but also the duration of the transient phase. Selecting $\alpha$ too small may result in generating congestion and not routing/processing data analytics.

VI-B3 Comparison with other algorithms

Given that this paper introduces a new problem, there are no other algorithms with which to compare. Hence, we use two intuitive alternatives: (i) a max-flow type algorithm [21] that makes routing/processing decisions based only on the network congestion (i.e., the Lagrange multipliers); and (ii) an algorithm that balances analytics performance and energy consumption (instead of network lifetime). Specifically, the first term in the objective in problem (10) is replaced with $\sum_{i\in\mathcal{N}}p_{i}(\bm{x},\bm{y})$.131313We assume that averages $\bm{\lambda}$ and $\bm{\mu}$ are known in the min-energy algorithm. Also, to compare the algorithms fairly, we fix the reward for processing analytics to $40$ mAP and evaluate their performance in terms of network lifetime gain (we do this by selecting $\eta$ and $\theta$ accordingly). Figure 4 shows the average data analytics reward for 50 different realizations (time-varying arrivals and link capacities). Parameter $\alpha$ is selected equal to $10^{-3}$. Observe that with the proposed algorithm, we obtain $48\%$ longer lifetime than with the algorithm that minimizes only the total energy consumption (min-energy), and a $900\%$ gain compared to the algorithm that only considers the network congestion (max-flow). Conclusion: minimizing the total energy consumption does not necessarily maximize the network lifetime. Algorithms that do not take into account the energy constraints can degrade the lifetime of the network significantly.

VII Conclusions

We have studied a crucial new problem in emerging IoT networks: how to allocate bandwidth and computation resources to data analytics tasks while considering the time the network can operate. The algorithm proposed can (i) balance between maximizing the network lifetime and the grade in which analytics are carried out; and (ii) operate without knowledge of the traffic arrivals or channel statistics. Our simulations seeded by real IoT device energy measurements, show that the network connectivity plays a crucial role in network lifetime maximization, that the algorithm can obtain both maximum network lifetime and maximum data analytics performance in addition to maximizing the joint objective, and that the algorithm increases the network lifetime by approximately 50% compared to an algorithm that minimizes the total energy consumption in the network.

VIII Appendix

VIII-A Preliminaries

To streamline exposition and due to space constraints, we assume in the analysis there is only one commodity. The extension to multiple commodities is nonetheless straightforward. Now, we write the Lagrangian as $L({x},{y},{\nu})=f(x,y)+\nu^{\top}(g(x,y)+\lambda),$ where $f(x,y)$ is the objective function in (13) and $g(x,y)=(g_{1}(x,y),\dots,g_{n}(x,y))$ with $g_{i}(x,y)=(\beta_{i}-1)y_{i}+\sum_{j\in\mathcal{N}_{i}}(x_{ji}-x_{ij})$, $i\in\{1,\dots,n\}$. That is, $g_{i}(x,y)+\lambda_{i}$ is the flow conservation constraint of node $i\in\mathcal{N}$ and $g(x,y)$ a collection of $n$ flow conservation constraints. Note we do not use bold notation and that $\nu^{\top}(g(x,y)+\lambda)$ is the inner product of vectors $\nu$ and $(g(x,y)+\lambda)$.

VIII-B Proof of Theorem 1

We have the dual (sub)gradient update $\nu[t+1]=\nu[t]+\alpha(g({x}[t],{y}[t])+\lambda)$ where ${x}[t]\in X[t]$ and ${y}[t]\in Y[t]$ are obtained by minimizing the Lagrangian for a fixed $\nu[t]$. We start by showing an upper bound on the objective function.

Lemma 2** (Objective upper bound).**

It holds

[TABLE]

Proof:

Since $x[t]$ and $y[t]$ are selected from $X[t]$ and $Y[t]$ to minimize the Lagrangian, we can write $\tilde{f}(x{[t]},y[t])+(\nu[t])^{\top}(g(x[t],y[t])+\lambda[t])=f(x{[t]},y[t])+\xi(x[t],y[t])+(\nu[t])^{\top}(g(x[t],y[t])+\lambda[t])\leq f(x^{\star}{[t]},y^{\star}[t])+\xi(x^{\star}[t],y^{\star}[t])+(\nu[t])^{\top}(g(x^{\star}[t],y^{\star}[t])+\lambda[t])$. Rearranging terms, summing from $t=1,\dots,T$, dividing by $T$, and using the fact that $|\xi(x[t],y[t])|\leq\epsilon_{2}$ by assumption for all $(x[t],y[t])\in(X[t],Y[t])$ yields the stated result. ∎

Next, we bound the first two terms in the RHS of (21).

Lemma 3**.**

The first term in the RHS of (21) is upper bounded by $\frac{\alpha}{2T}\sum_{t=1}^{T}\sigma[t]$ where $\sigma[t]:=\|g(x[t],y[t])+\lambda[t]\|_{2}^{2}$.

Proof:

Let $\theta\in{\mathbf{R}}^{n}$. From the standard dual subgradient update, we have that $\|\nu[t+1]-\theta\|_{2}^{2}=\|\nu[t]+\alpha(g(x[t],y[t])+\lambda[t])-\theta\|_{2}^{2}=\|\nu[t]-\theta\|_{2}^{2}+\alpha^{2}\|g(x[t],y[t])+\lambda[t]\|_{2}^{2}+2\alpha(\nu[t]-\theta)^{\top}(g(x[t],y[t])+\lambda[t])\leq\|\nu[t]-\theta\|_{2}^{2}+\alpha^{2}\sigma[t]+2\alpha(\nu[t]-\theta)^{\top}(g(x[t],y[t])+\lambda[t]).$ Parameter $\sigma[t]$ is bounded since $X[t]$, $Y[t]$, $\lambda[t]$ are bounded for all $t\in{\mathbf{Z}}_{+}$. Next, rearrange terms, apply the expansion recursively for $t=1,\dots,T$ to obtain $-2\alpha\sum_{t=1}^{T}(\nu[t]-\theta)^{\top}(g(x[t],y[t])+\lambda[t])\leq\alpha^{2}\sum_{t=1}^{T}\sigma[t]+\|\nu[1]-\theta\|_{2}^{2}-\|\nu[t]-\theta\|_{2}^{2}$. Drop the third term in the RHS of the last equation since it is nonnpositive, let $\theta=0$ and $\nu[1]=0$, and divide across by $2\alpha T$ to obtain the stated bound. ∎

Lemma 4**.**

The second term in the RHS of (21) is equal to zero on expectation.

Proof:

Take expectations with respect to random variable $X[t]$, $Y[t]$, $\lambda[t]$, and write ${\mathbf{E}}(\frac{1}{T}\sum_{t=1}^{T}(\nu[t])^{\top}(g(x^{\star}[t],y^{\star}[t])+\lambda[t]))\stackrel{{\scriptstyle\text{(a)}}}{{=}}\frac{1}{T}\sum_{t=1}^{T}(\nu[t])^{\top}{\mathbf{E}}(g(x^{\star}[t],y^{\star}[t])+\lambda[t])\stackrel{{\scriptstyle\text{(b)}}}{{=}}0$ where (a) follows from the linearity of the expectation, and (b) since ${\mathbf{E}}(g(x^{\star}[t],y^{\star}[t])+\lambda[t])={\mathbf{E}}(g(x^{\star}[t],y^{\star}[t]))+{\mathbf{E}}(\lambda[t])=g(x^{\star},y^{\star})+\lambda=0$ (i.e., at the optimum the problem must be feasible). Note that this is always case since random variables $X[t]$, $Y[t]$, $\lambda[t]$ do not depend on $\nu[t]$ for all $t\in{\mathbf{Z}}_{+}$. ∎

We are now in position to prove claim (i). Take expectations in the bound obtain in Lemma 2. The first term can be upper bounded by Lemma 3 and the second with Lemma 4. By letting ${\mathbf{E}}(\sigma[t])\leq 2\epsilon_{1}$ for all $t\in{\mathbf{Z}}_{+}$ the stated result follows.

Lemma 5** (Feasibility).**

${\mathbf{E}}(g(\bar{x},\bar{y})+\lambda)=0$* converges to a feasible point asymptotically as $T\to\infty$.*

Proof:

Recall $\nu[t+1]=\nu[t]+\alpha(g(x[t],y[t])+\lambda[t]))$. Rearrange terms and apply the iteration recursively to obtain $\nu[t+1]-\nu[1]=\sum_{t=1}^{T}\alpha(g(x[t],y[t])+\lambda[t])).$ Dividing by $T$ and using the fact that $\nu[1]=0$ we have $\frac{1}{T}\sum_{t=1}^{T}g(x[t],y[t])+\lambda[t]=g(\bar{x},\bar{y})+\bar{\lambda}={\nu[t]}/({\alpha T})$ which implies that $(g(\bar{x},\bar{y})+\bar{\lambda})\to 0$ as $T\to\infty$ if $\nu[t]$ is bounded. The latter will be the case, on expectation, when $\nu[t]$ is a nonnegative process (nodes do not pre-serve/process data) and the flow conservation constraints are not tight: there exists a $\chi\succ 0$ and $(\hat{x},\hat{y})$ such that $g(\hat{x},\hat{y})+\lambda+\chi=0$ holds. ∎