A New Backpressure Algorithm for Joint Rate Control and Routing with Vanishing Utility Optimality Gaps and Finite Queue Lengths

TL;DR

This paper introduces a novel backpressure algorithm that achieves near-optimal utility with finite queue lengths, overcoming the traditional utility-delay tradeoff in multi-hop networks.

Contribution

It proposes a new backpressure algorithm that guarantees vanishing utility gaps while maintaining bounded queue lengths, a significant improvement over existing methods.

Findings

Utility gap approaches zero as the algorithm runs

Queue lengths remain bounded by a finite constant

The method uses a new convex programming approach

Abstract

The backpressure algorithm has been widely used as a distributed solution to the problem of joint rate control and routing in multi-hop data networks. By controlling a parameter $V$ in the algorithm, the backpressure algorithm can achieve an arbitrarily small utility optimality gap. However, this in turn brings in a large queue length at each node and hence causes large network delay. This phenomenon is known as the fundamental utility-delay tradeoff. The best known utility-delay tradeoff for general networks is $[O(1/V), O(V)]$ and is attained by a backpressure algorithm based on a drift-plus-penalty technique. This may suggest that to achieve an arbitrarily small utility optimality gap, the existing backpressure algorithms necessarily yield an arbitrarily large queue length. However, this paper proposes a new backpressure algorithm that has a vanishing utility optimality gap, so utility converges to exact optimality as the algorithm keeps running, while queue lengths are bounded throughout by a finite constant. The technique uses backpressure and drift concepts with a new method for convex programming.

Equations

$$\underset{x_f,{\mu }_l^{(f)}}{\max }$$

$$x_f{1}_{\{n=\text{Src}(f)\}}+\sum _{l\in \mathcal{I}(n)}{\mu }_l^{(f)}\le \sum _{l\in \mathcal{O}(n)}{\mu }_l^{(f)},\forall f\in \mathcal{F},\forall n\in \mathcal{N}\setminus \{\text{Dst}(f)\}$$

$$\sum _{f\in \mathcal{F}}{\mu }_l^{(f)}\le C_l,\forall l\in \mathcal{L},$$

$${\mu }_l^{(f)}\ge 0,\forall l\in \mathcal{L},\forall f\in {\mathcal{S}}_l,$$

$${\mu }_l^{(f)}=0,\forall l\in \mathcal{L},\forall f\in \mathcal{F}\setminus {\mathcal{S}}_l,$$

$$x_f\in \text{mbox}dom(U_f),\forall f\in \mathcal{F}$$

$$q({\mathbf{\lambda }}^{\ast })=\sup \{\text{eqref}eq:opt-obj:\text{eqref}eq:opt-flow-balance-cons\text{-}\text{eqref}eq:opt-rate-nonnegative\}$$

$$Y_n^{(f)}[t+1]=$$

$$Z_n^{(f)}[t+1]\le$$

$$Q_n^{(f)}[t+1]=$$

$${\hat{Q}}_n^{(f)}[t+1]=$$

$$\frac{1}{t}\sum _{\tau =0}^{t-1}\sum _{f\in \mathcal{F}}{U}_f(x_f[t])\ge \sum _{f\in \mathcal{F}}{U}_f(x_f^{\ast })-O(1/t),\forall t$$

$$\sum_{f\in\mathcal{F}}U_{f}\big{(}\frac{1}{t}\sum_{\tau=0}^{t-1}x_{f}[\tau]\big{)}\geq\sum_{f\in\mathcal{F}}U_{f}(x_{f}^{\ast})-O(1/t),\forall t$$

$$W_n^{(f)}[t]=$$

$$\underset{x_f}{\max }$$

$$x_f\in \text{mbox}dom(U_f)$$

$$\underset{{\mu }_{(n,m)}^{(f)}}{\max }$$

$$\sum _{f\in \mathcal{F}}{\mu }_{(n,m)}^{(f)}\le C_{(n,m)}$$

$${\mu }_{(n,m)}^{(f)}\ge 0,\forall f\in {\mathcal{S}}_{(n,m)}$$

$${\mu }_{(n,m)}^{(f)}=0,\forall f\notin {\mathcal{S}}_{(n,m)}$$

$${\hat{x}}_f$$

$$\min$$

$$\sum _{k=1}^K{z}_k\le b$$

$$z_k\ge 0,\forall k\in \{1,2,\dots ,K\}$$

$$\displaystyle\mathbf{y}_{n}^{(f)}=\left\{\begin{array}[]{ll}~{}[x_{f};\mu_{l}^{(f)}]_{l\in\mathcal{I}(n)\cup\mathcal{O}(n)}&\text{if}~{}n=\text{Src}(f),\\ \ [\mu_{l}^{(f)}]_{l\in\mathcal{I}(n)\cup\mathcal{O}(n)}&\text{else},\end{array}\right.$$

$$g_n^{(f)}({\mathbf{y}}_n^{(f)})=x_f{1}_{\{n=\text{Src}(f)\}}+\sum _{l\in \mathcal{I}(n)}{\mu }_l^{(f)}-\sum _{l\in \mathcal{O}(n)}{\mu }_l^{(f)}$$

$$g_n^{(f)}({\mathbf{y}}_n^{(f)})\le 0,\forall f\in \mathcal{F},\forall n\in \mathcal{N}\setminus \{\text{Dst}(f)\}.$$

$${\beta }_n\le \sqrt{d_n+1}.$$

$$Q_n^{(f)}[t+1]=Q_n^{(f)}[t]+g_n^{(f)}({\mathbf{y}}_n^{(f)}[t]),$$

$$W_n^{(f)}[t]=Q_n^{(f)}[t]+g_n^{(f)}({\mathbf{y}}_n^{(f)}[t-1]).$$

$$\displaystyle L(t)=\frac{1}{2}\sum_{f\in\mathcal{F}}\sum_{n\in\mathcal{N}\setminus\text{Dst}(f)}\big{(}Q_{n}^{(f)}[t]\big{)}^{2}$$

$$\displaystyle\sum_{f\in\mathcal{F}}\sum_{n\in\mathcal{N}\setminus\text{Dst}(f)}\big{(}\cdot\big{)}\overset{\Delta}{=}\sum_{\begin{subarray}{c}f\in\mathcal{F},\\ n\in\mathcal{N}\setminus\text{Dst}(f)\end{subarray}}\big{(}\cdot\big{)}.$$

$$\Delta [t]=L(t+1)-L(t).$$

$$\displaystyle\Delta[t]=\sum_{\begin{subarray}{c}f\in\mathcal{F},\\ n\in\mathcal{N}\setminus\text{Dst}(f)\end{subarray}}\Big{(}Q_{n}^{(f)}[t]g_{n}^{(f)}(\mathbf{y}_{n}^{f}[t])+\frac{1}{2}\big{(}g_{n}^{(f)}(\mathbf{y}_{n}^{f}[t])\big{)}^{2}\Big{)}.$$

$$\displaystyle\frac{1}{2}\big{(}Q_{n}^{(f)}[t+1]\big{)}^{2}-\frac{1}{2}\big{(}Q_{n}^{(f)}[t]\big{)}^{2}$$

$$\stackrel{(a)}{=}$$

Full text

I Introduction

In multi-hop data networks, the problem of joint rate control and routing is to accept data into the network to maximize certain utilities and to make routing decisions at each node such that all accepted data are delivered to intended destinations without overflowing any queue in intermediate nodes. The original backpressure algorithm proposed in the seminal work [1] by Tassiulas and Ephremides addresses this problem by assuming that incoming data are given and are inside the network stability region and develops a routing strategy to deliver all incoming data without overflowing any queue. In the context of [1], there is essentially no utility maximization consideration in the network. The backpressure algorithm is further extended by a drift-plus-penalty technique to deal with data network with both utility maximization and queue stability considerations [2, 3, 4]. Alternative extensions for both utility maximization and queue stabilization are developed in [5, 6, 7, 8]. The above extended backpressure algorithms have different dynamics and/or may yield different utility-delay tradeoff results. However, all of them rely on “backpressure” quantities, which are the differential backlogs between neighboring nodes.

It has been observed in [9, 5, 7, 10] that the drift-plus-penalty and other alternative algorithms can be interpreted as first order Lagrangian dual type methods for constrained optimization. In addition, these backpressure algorithms follow certain fundamental utility-delay tradeoffs. For instance, the primal-dual type backpressure algorithm in [5] achieves an $O(1/V)$ utility optimality gap with an $O(V^{2})$ queue length, where $V$ is an algorithm parameter. By controlling parameter $V$, a small utility optimality gap is available only at the cost of a large queue length. The drift-plus-penalty backpressure algorithm [4], which has the best utility-delay tradeoff among all existing first order Lagrangian dual type methods for general networks, can only achieve an $O(1/V)$ utility optimality gap with an $O(V)$ queue length. Under certain restrictive assumptions over the network, a better $[O(1/V),O(\log(V))]$ tradeoff is achieved via an exponential Lyapunov function in [11], and an $[O(1/V),O(\log^{2}(V))]$ tradeoff is achieved via a LIFO-backpressure algorithm in [12]. The existing utility-delay tradeoff results seem to suggest that a large queueing delay is unavoidable if a small utility optimality gap is demanded.

Recently, there have been many attempts in obtaining new variations of backpressure algorithms by applying Newton’s method to the Lagrangian dual function. In the recent work [10], the authors develop a Newton’s method for joint rate control and routing. However, the utility-delay tradeoff in [10] is still $[O(1/V),O(V^{2})]$; and the algorithm requires a centralized projection step (although Newton directions can be approximated in a distributed manner). Work [13] considers a network flow control problem where the path of each flow is given (and hence there is no routing part in the problem), and proposes a decentralized Newton based algorithm for rate control. Work [14] considers network routing without an end-to-end utility and only shows the stability of the proposed Newton based backpressure algorithm. All of the above Netwon’s method based algorithms rely on distributed approximations for the inverse of Hessians, whose computations still require certain coordinations for the local information updates and propagations and do not scale well with the network size. In contrast, the first order Lagrangian dual type methods do not need global network topology information. Rather, each node only needs the queue length information of its neighbors.

This paper proposes a new first order Lagrangian dual type backpressure algorithm that is as simple as the existing algorithms in [4, 5, 7] but has a better utility-delay tradeoff. The new backpressue algorithm achieves a vanishing utility optimality gap that decays like $O(1/t)$, where $t$ is the number of iterations. It also guarantees that the queue length at each node is always bounded by a fixed constant of the same order as the optimal Lagrange multiplier of the network optimization problem. This improves on the utility-delay tradeoffs of prior work. In particular, it improves the $[O(1/V),O(V^{2})]$ utility-delay tradeoff in [5] and the $[O(1/V),O(V)]$ utility-delay tradeoff of the drift-plus-penalty algorithm in [4], both of which yield an unbounded queue length to have a vanishing utility optimality gap. The new backpressure algorithm differs from existing first order backpressure algorithms in the following aspects:

The “backpressure” quantities in this paper are with respect to newly introduced weights. These are different from queues used in other backpressure algorithms, but can still be locally tracked and updated. 2. 2.

The rate control and routing decision rule involves a quadratic term that is similar to a term used in proximal algorithms [15].

Note that the benefit of introducing a quadratic term in network optimization has been observed in [16]. Work [16] considers a network utility maximization problem with given routing paths that is a special case of the problem treated in this paper. The algorithm of [16] considers a fixed set of predetermined paths for each session and does not scale well when treating all (typically exponentially many) possible paths of a general network. The algorithm proposed in [16] is not a backpressure type and hence is fundamentally different from ours. For example, the algorithm in [16] needs to update the primal variables (source session rates for each path) at least twice per iteration, while our algorithm only updates the primal variables (source session rates and link session rates) once per iteration. The prior work [16] shows that the utility optimality gap is asymptotically zero without analyzing the decay rate, while this paper shows the utility optimality gap decays like $O(1/t)$.

II System Model and Problem Formulation

Consider a slotted data network with normalized time slots $t\in\{0,1,2,\ldots\}$. This network is represented by a graph $\mathcal{G}=(\mathcal{N},\mathcal{L})$, where $\mathcal{N}$ is the set of nodes and $\mathcal{L}\subseteq\mathcal{N}\times\mathcal{N}$ is the set of directed links. Let $|\mathcal{N}|=N$ and $|\mathcal{L}|=L$. This network is shared by $F$ end-to-end sessions denoted by a set $\mathcal{F}$. For each end-to-end session $f\in\mathcal{F}$, the source node $\text{Src}(f)$ and destination node $\text{Dst}(f)$ are given but the routes are not specified. Each session $f$ has a continuous and concave utility function $U_{f}(x_{f})$ that represents the “satisfaction” received by accepting $x_{f}$ amount of data for session $f$ into the network at each slot. Unlike [5, 10] where $U_{f}(\cdot)$ is assumed to be differentiable and strongly concave, this paper considers general concave utility functions $U_{f}(\cdot)$, including those that are neither differentiable nor strongly concave. Formally, each utility function $U_{f}$ is defined over an interval $\mbox{dom}(U_{f})$, called the domain of the function. It is assumed throughout that either $\mbox{dom}(U_{f})=[0,\infty)$ or $\mbox{dom}(U_{f})=(0,\infty)$, the latter being important for proportionally fair utilities [17] $U_{f}(x)=\log(x)$ that have singularities at $x=0$ .

Denote the capacity of link $l$ as $C_{l}$ and assume it is a fixed and positive constant.111As stated in [10], this is a suitable model for wireline networks and wireless networks with fixed transmission power and orthogonal channels. Define $\mu_{l}^{(f)}$ as the amount of session $f$’s data routed at link $l$ that is to be determined by our algorithm. Note that in general, the network may be configured such that some session $f$ is forbidden to use link $l$. For each link $l$, define $\mathcal{S}_{l}\subseteq\mathcal{F}$ as the set of sessions that are allowed to use link $l$. The case of unrestricted routing is treated by defining $\mathcal{S}_{l}=\mathcal{F}$ for all links $l$.

Note that if $l=(n,m)$ with $n,m\in\mathcal{N}$, then $\mu_{l}^{(f)}$ and $C_{l}$ can also be respectively written as $\mu_{(n,m)}^{(f)}$ and $C_{(n,m)}$. For each node $n\in\mathcal{N}$, denote the sets of its incoming links and outgoing links as $\mathcal{I}(n)$ and $\mathcal{O}(n)$, respectively. Note that $x_{f},\forall f\in\mathcal{F}$ and $\mu_{l}^{(f)},\forall l\in\mathcal{L},\forall f\in\mathcal{F}$ are the decision variables of a joint rate control and routing algorithm. If the global network topology information is available, the optimal joint rate control and routing can be formulated as the following multi-commodity network flow problem:

[TABLE]

where $\mathbf{1}_{\{\cdot\}}$ is an indicator function; (2) represents the node flow conservation constraints relaxed by replacing the equality with an inequality, meaning that the total rate of flow $f$ into node $n$ is less than or equal to the total rate of flow $f$ out of the node (since, in principle, we can always send fake data for departure links when the inequality is loose); and (3) represents link capacity constraints. Note that for each flow $f$, there is no constraint (2) at its destination node $\text{Dst}(f)$ since all incoming data are consumed by this node.

The above formulation includes network utility maximization with fixed paths as special cases. In the case when each session only has one single given path, e.g., the network utility maximization problem considered in [18], we could modify the sets $\mathcal{S}_{l}$ used in constraints (4) and (5) to reflect this fact. For example, if link $l_{1}$ is only used for sessions $f_{1}$ and $f_{2}$, then $\mathcal{S}_{l_{1}}=\{f_{1},f_{2}\}$. Similarly, the case [16] where each flow is restricted to using links from a set of predefined paths can be treated by modifying the sets $\mathcal{S}_{l}$ accordingly. See Appendix A for more discussions.

The solution to problem (1)-(6) corresponds to the optimal joint rate control and routing. However, to solve this convex program at a single computer, we need to know the global network topology and the solution is a centralized one, which is not practical for large data networks. As observed in [9, 5, 7, 10], various versions of backpressure algorithms can be interpreted as distributed solutions to problem (1)-(6) from first order Lagrangian dual type methods.

Assumption 1

(Feasibility) Problem (1)-(6) has at least one optimal solution vector $[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$.

Assumption 2

(Existence of Lagrange multipliers) Assume the convex program (1)-(6) has Lagrange multipliers attaining the strong duality. Specifically, define convex set $\mathcal{C}=\{[x_{f};\mu_{l}^{(f)}]_{f\in\mathcal{F},l\in\mathcal{L}}:\eqref{eq:opt-link-capacity-cons}\text{-}\eqref{eq:opt-rate-nonnegative}~{}\text{hold}\}$. Assume there exists a Lagrange multiplier vector $\boldsymbol{\lambda}^{\ast}=[\lambda_{n}^{(f),\ast}]_{f\in\mathcal{F},n\in\mathcal{N}\setminus\{\text{Dst}(f)\}}\geq\mathbf{0}$ such that

[TABLE]

where q(\boldsymbol{\lambda})=\sup_{[x_{f};\mu_{l}^{(f)}]\in\mathcal{C}}\big{\{}\sum_{f\in\mathcal{F}}U_{f}(x_{f})-\sum_{f\in\mathcal{F}}\sum_{n\in\mathcal{N}\setminus\{\text{Dst}(f)\}}\lambda_{n}^{(f)}\big{[}x_{f}\mathbf{1}_{\{n=\text{Src}(f)\}}+\sum_{l\in\mathcal{I}(n)}\mu_{l}^{(f)}-\sum_{l\in\mathcal{O}(n)}\mu_{l}^{(f)}\big{]}\big{\}} is the Lagrangian dual function of problem (1)-(6) by treating (3)-(6) as a convex set constraint.

Assumptions 1 and 2 hold in most cases of interest. For example, Slater’s condition guarantees Assumption 2. Since the constraints (2)-(6) are linear, Proposition 6.4.2 in [19] ensures that Lagrange multipliers exist whenever constraints (2)-(6) are feasible and when the utility functions $U_{f}$ are either defined over open sets (such as $U_{f}(x)=\log(x)$ with $\mbox{dom}(U_{f})=(0,\infty)$) or can be concavely extended to open sets, meaning that there is an $\epsilon>0$ and a concave function $\widetilde{U}_{f}:(-\epsilon,\infty)\rightarrow\mathbb{R}$ such that $\widetilde{U}_{f}(x)=U_{f}(x)$ whenever $x\geq 0$.222If $\mbox{dom}(U_{f})=[0,\infty)$, such concave extension is possible if the right-derivative of $U_{f}$ at $x=0$ is finite (such as for $U_{f}(x)=\log(1+x)$ or $U_{f}(x)=\min[x,3]$). Such an extension is impossible for the example $U_{f}(x)=\sqrt{x}$ because the slope is infinite at $x=0$. Nevertheless, Lagrange multipliers often exist even for these utility functions, such as when Slater’s condition holds [19].

Fact 1

(Replacing inequality with equality) If Assumption 1 holds, problem (1)-(6) has an optimal solution vector $[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$ such that all constraints (2) take equalities.

Proof:

Note that each $\mu_{l}^{(f)}$ can appear on the left side in at most one constraint (2) and appear on the right side in at most one constraint (2). Let $[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$ be an optimal solution vector such that at least one inequality constraint (2) is loose. Note that we can reduce the value of $\mu_{l}^{(f),\ast}$ on the right side of a loose (2) until either that constraint holds with equality, or until $\mu_{l}^{(f),\ast}$ reduces to [math]. The objective function value does not change, and no constraints are violated. We can repeat the process until all inequality constraints (2) are tight. ∎

III The New Backpresure Algorithm

III-A Discussion of Various Queueing Models

At each node, an independent queue backlog is maintained for each session. At each slot $t$, let $x_{f}[t]$ be the source session rates; and let $\mu_{l}^{(f)}[t]$ be the link session rates. Some prior work enforces the constraint (2) via virtual queues $Y_{n}^{(f)}[t]$ of the following form:

[TABLE]

While this virtual equation is a meaningful approximation, it differs from reality in that new injected data are allowed to be transmitted immediately, or equivalently, a single packet is allowed to enter and leave many nodes within the same slot. Further, there is no clear connection between the virtual queues $Y_{n}^{(f)}[t]$ in (7) and the actual queues in the network. Indeed, it is easy to construct examples that show there can be an arbitrarily large difference between the $Y_{n}^{(f)}[t]$ value in (7) and the physical queue size in actual networks (see Appendix B).

An actual queueing network has queues $Z_{n}^{(f)}[t]$ with the following dynamics:

[TABLE]

This is faithful to actual queue dynamics and does not allow data to be retransmitted over multiple hops in one slot. Note that (8) is an inequality because the new arrivals from other nodes may be strictly less than $\sum_{l\in\mathcal{I}(n)}\mu_{l}^{(f)}[t]$ because those other nodes may not have enough backlog to send. The model (8) allows for any decisions to be made to fill the transmission values $\mu_{l}^{(f)}[t]$ in the case that $Z_{n}^{(f)}[t]\leq\sum_{l\in\mathcal{O}(n)}\mu_{l}^{(f)}[t]$, provided that (8) holds.

This paper develops an algorithm that converges to the optimal utility defined by problem (1)-(6), and that produces worst-case bounded queues on the actual queueing network, that is, with actual queues that evolve as given in (8). To begin, it is convenient to introduce the following virtual queue equation

[TABLE]

where $Q_{n}^{(f)}[t]$ represents a virtual queue value associated with session $f$ at node $n$. At first glance, this model (9) appears to be only an approximation, perhaps even a worse approximation than (7), because it allows the $Q_{n}^{(f)}[t]$ values to be negative. Indeed, we use $Q_{n}^{(f)}[t]$ only as virtual queues to inform the algorithm and do not treat them as actual queues. However, this paper shows that using these virtual queues to choose the $\boldsymbol{\mu}[t]$ decisions ensures not only that the desired constraints (2) are satisfied, but that the resulting $\boldsymbol{\mu}[t]$ decisions create bounded queues $Z_{n}^{(f)}[t]$ in the actual network, where the actual queues evolve according to (8). In short, our algorithm can be faithfully implemented with respect to actual queueing networks, and converges to exact optimality on those networks.

The next lemma shows that if an algorithm can guarantee virtual queues $Q_{n}^{(f)}[t]$ defined in (9) are bounded, then actual physical queues satisfying (8) are also bounded.

Lemma 1

Consider a network flow problem described by problem (1)-(6). For all $l\in\mathcal{L}$ and $f\in\mathcal{F}$, let $\mu_{l}^{(f)}[t],x_{f}[t]$ be decisions yielded by a dynamic algorithm. Suppose $Y_{n}^{(f)}[t]$, $Z_{n}^{(f)}[t]$, $Q_{n}^{(f)}[t]$ evolve by (7)-(9) with initial conditions $V_{n}^{(f)}[0]=Z_{n}^{(f)}[0]=Q_{n}^{(f)}[0]=0$. If there exists a constant $B>0$ such that $|Q_{n}^{(f)}[t]|\leq B,\forall t$, then

$Z_{n}^{(f)}[t]\leq 2B+\sum_{l\in\mathcal{O}(n)}C_{l}$* for all $t\in\{0,1,2,\ldots\}$.* 2. 2.

$Y_{n}^{(f)}[t]\leq 2B+\sum_{l\in\mathcal{O}(n)}C_{l}$* for all $t\in\{0,1,2,\ldots\}$.*

Proof:

Fix $f\in\mathcal{F},n\in\mathcal{N}\setminus\{\text{Dst}(f)\}$. Define an auxiliary virtual queue $\widehat{Q}_{n}^{(f)}[t]$ that is initialized by $\widehat{Q}_{n}^{(f)}[0]=B+\sum_{l\in\mathcal{O}(n)}C_{l}$ and evolves according to (9). It follows that $\widehat{Q}_{n}^{(f)}[t]=Q_{n}^{(f)}[t]+B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$. Since $Q_{n}^{(f)}[t]\geq-B,\forall t$ by assumption, we have $\widehat{Q}_{n}^{(f)}[t]\geq\sum_{l\in\mathcal{O}(n)}C_{l}\geq\sum_{l\in\mathcal{O}(n)}\mu_{l}^{(f)}[t],\forall t$. This implies that $\widehat{Q}_{n}^{(f)}[t]$ also satisfies:

[TABLE]

which is identical to (8) except the inequality is replaced by an equality. Since $Z_{n}^{(f)}[0]=0<\widehat{Q}_{n}^{(f)}[0]$; and $\widehat{Q}_{n}^{(f)}[t]$ satisfies (10), by inductions, $Z_{n}^{(f)}[t]\leq\widehat{Q}_{n}^{(f)}[t],\forall t$.

Since $\widehat{Q}_{n}^{(f)}[t]=Q_{n}^{(f)}[t]+B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$ and $Q_{n}^{(f)}[t]\leq B,\forall t$, we have $\widehat{Q}_{n}^{(f)}[t]\leq 2B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$. It follows that $Z_{n}^{(f)}[t]\leq 2B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$. 2. 2.

The proof of part (2) is similar and is in Appendix C.

∎

III-B The New Backpressure Algorithm

In this subsection, we propose a new backpressure algorithm that yields source session rates $x_{f}[t]$ and link session rates $\mu_{l}^{(f)}[t]$ at each slot such that the physical queues for each session at each node are bounded by a constant and the time average utility satisfies

[TABLE]

where $x_{f}^{\ast}$ are from the optimal solution to (1)-(6). Note that Jensen’s inequality further implies that

[TABLE]

The new backpressure algorithm is described in Algorithm 1. Similar to existing backpressure algorithms, the updates in Algorithm 1 at each node $n$ are fully distributed and only depend on weights at itself and its neighbor nodes. Unlike existing backpressure algorithms, the weights used to update decision variables $x_{f}[t]$ and $\mu_{l}^{(f)}[t]$ are not the virtual queues $Q_{n}^{(f)}[t]$ themselves, rather, they are augmented values $W_{n}^{(f)}[t]$ equal to the sum of the virtual queues and the amount of net injected data in the previous slot $t-1$. In addition, the updates involve an additional quadratic term, which is similar to a term used in proximal algorithms [15].

III-C Almost Closed-Form Updates in Algorithm 1

This subsection shows the decisions $x_{f}[t]$ and $\mu_{l}^{(f)}[t]$ in Algorithm 1 have either closed-form solutions or “almost” closed-form solutions at each iteration $t$.

Lemma 2

Let $\hat{x}_{f}\equiv x_{f}[t]$ denote the solution to (12)-(13).

Suppose $\mbox{dom}(U_{f})=[0,\infty)$ and $U_{f}(x_{f})$ is differentiable. Let $h(x_{f})=U_{f}^{\prime}(x_{f})-2\alpha_{n}x_{f}+2\alpha_{n}x_{f}[t-1]-W_{n}^{(f)}[t]$. If $h(0)<0$, then $\hat{x}_{f}=0$; otherwise $\hat{x}_{f}$ is the root to the equation $h(x_{f})=0$ and can be found by a bisection search. 2. 2.

Suppose $\mbox{dom}(U_{f})=(0,\infty)$ and $U_{f}(x_{f})=w_{f}\log(x_{f})$ for some weight $w_{f}>0$. Then:

[TABLE]

Proof:

Omitted for brevity. ∎

The problem (14)-(17) can be represented as follows by eliminating $\mu_{(n,m)}^{(f)},f\not\in\mathcal{S}_{(n,m)}$, completing the square and replacing maximization with minimization. (Note that $K=|\mathcal{S}_{(n,m)}|\leq|\mathcal{F}|$.)

[TABLE]

Lemma 3

The solution to problem (18)-(20) is given by $z_{k}^{\ast}=\max\{0,a_{k}-\theta^{\ast}\},\forall k\in\{1,2,\ldots,K\}$ where $\theta^{\ast}\geq 0$ can be found either by a bisection search (See Appendix D) or by Algorithm 2 with complexity $O(K\log K)$.

Proof:

A similar problem where (19) is replaced with an equality constraint in considered in [20]. The optimal solution to this quadratic program is characterized by its KKT condition and a corresponding algorithm can be developed to obtain its KKT point. A complete proof is presented in Appendix D. ∎

Note that step (3) in Algorithm 2 has complexity $O(K)$ and hence the overall complexity of Algorithm 2 is dominated by the sorting step (2) with complexity $O(K\log(K))$.

IV Performance Analysis of Algorithm 1

IV-A Basic Facts from Convex Analysis

Definition 1** (Lipschitz Continuity)**

Let $\mathcal{Z}\subseteq\mathbb{R}^{n}$ be a convex set. Function $h:\mathcal{Z}\rightarrow\mathbb{R}^{m}$ is said to be Lipschitz continuous on $\mathcal{Z}$ with modulus $\beta$ if there exists $\beta>0$ such that $\|h(\mathbf{z}_{1})-h(\mathbf{z}_{2})\|\leq\beta\|\mathbf{z}_{1}-\mathbf{z}_{2}\|$ for all $\mathbf{z}_{1},\mathbf{z}_{2}\in\mathcal{Z}$.

Definition 2** (Strongly Concave Functions)**

Let $\mathcal{Z}\subseteq\mathbb{R}^{n}$ be a convex set. Function $h$ is said to be strongly concave on $\mathcal{Z}$ with modulus $\alpha$ if there exists a constant $\alpha>0$ such that $h(\mathbf{z})+\frac{1}{2}\alpha\|\mathbf{z}\|^{2}$ is concave on $\mathcal{Z}$.

By the definition of strongly concave functions, it is easy to show that if $h(\mathbf{z})$ is concave and $\alpha>0$, then $h(\mathbf{z})-\alpha\|\mathbf{z}-\mathbf{z}_{0}\|^{2}$ is strongly concave with modulus $2\alpha$ for any constant $\mathbf{z}_{0}$.

Lemma 4

Let $\mathcal{Z}\subseteq\mathbb{R}^{n}$ be a convex set. Let function $h$ be strongly concave on $\mathcal{Z}$ with modulus $\alpha$ and $\mathbf{z}^{opt}$ be a global maximum of $h$ on $\mathcal{Z}$. Then, $h(\mathbf{z}^{opt})\geq h(\mathbf{z})+\frac{\alpha}{2}\|\mathbf{z}^{opt}-\mathbf{z}\|^{2}$ for all $\mathbf{z}\in\mathcal{Z}$.

IV-B Preliminaries

Define column vector $\mathbf{y}=[x_{f};\mu_{l}^{(f)}]_{f\in\mathcal{F},l\in\mathcal{L}}$. For each $f\in\mathcal{F},n\in\mathcal{N}\setminus\{\text{Dst}(f)\}$, define column vector

[TABLE]

which is composed by the control actions appearing in each constraint (2); and introduce a function with respect to $\mathbf{y}_{n}^{(f)}$ as

[TABLE]

Thus, constraint (2) can be rewritten as

[TABLE]

Note that each vector $\mathbf{y}_{n}^{(f)}$ is a subvector of $\mathbf{y}$ and has length $d_{n}+1$ where $d_{n}$ is the degree of node $n$ (the total number of outgoing links and incoming links) if node $n$ is the source of session $f$; and has length $d_{n}$ if node $n$ is not the source of session $f$.

Fact 2

Each function $g_{n}^{(f)}(\cdot)$ defined in (24) is Lipschitz continuous with respect to vector $\mathbf{y}_{n}^{(f)}$ with modulus

[TABLE]

where $d_{n}$ is the degree of node $n$.

Proof:

This fact can be easily shown by noting that each $g_{n}^{(f)}(\mathbf{y}_{n}^{(f)})$ is a linear function with respect to vector $\mathbf{y}_{n}^{(f)}$ and has at most $d_{n}+1$ non-zero coefficients that are equal to $\pm 1$. ∎

Note that virtual queue update equation (9) can be rewritten as:

[TABLE]

and weight update equation (11) can be rewritten as:

[TABLE]

Define

[TABLE]

and call it a Lyapunov function. In the remainder of this paper, double summations are often written compactly as a single summation, e.g.,

[TABLE]

Define the Lyapunov drift as

[TABLE]

The following lemma follows directly from equation (25).

Lemma 5

At each iteration $t\in\{0,1,\ldots\}$ in Algorithm 1, the Lyapunov drift is given by

[TABLE]

Proof:

Fix $f\in\mathcal{F}$ and $n\in\mathcal{N}\setminus\text{Dst}(f)$, we have

[TABLE]

where (a) follows from (25).

By the definition of $\Delta[t]$, we have

[TABLE]

where (a) follows from (29). ∎

Define $f(\mathbf{y})=\sum_{f\in\mathcal{F}}U_{f}(x_{f})$. At each time $t$, consider choosing a decision vector $\mathbf{y}[t]$ that includes elements in each subvector $\mathbf{y}_{n}^{(f)}[t]$ to solve the following problem:

[TABLE]

The expression (30) is a modified drift-plus-penalty expression. Unlike the standard drift-plus-penalty expressions from [4], the above expression uses weights $W_{n}^{(f)}[t]$, which arguments each $Q_{n}^{(f)}[t]$ by $g_{n}^{(f)}(\mathbf{y}_{n}^{(f)}[t-1])$, rather than virtual queues $Q_{n}^{(f)}[t]$. It also includes a “prox”-like term that penalizes deviation from the previous $\mathbf{y}[t-1]$ vector. This results in the novel backpressure-type algorithm of Algorithm 1. Indeed, the decisions in Algorithm 1 were derived as the solution to the above problem (30)-(31). This is formalized in the next lemma.

Lemma 6

At each iteration $t\in\{0,1,\ldots\}$, the action $\mathbf{y}[t]$ jointly chosen in Algorithm 1 is the solution to problem (30)-(31).

Proof:

The proof involves collecting terms associated with the $x_{f}[t]$ and $\mu_{l}^{(f)}[t]$ decisions. See Appendix E for details. ∎

Furthermore, the next lemma summarizes that the action $\mathbf{y}[t]$ jointly chosen in Algorithm 1 provides a lower bound for the drift-plus-penalty expression at each iteration $t\in\{0,1,\ldots\}$.

Lemma 7

Let $\mathbf{y}^{\ast}=[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$ be an optimal solution to problem (1)-(6) given in Fact 1, i.e., $g_{n}^{(f)}(\mathbf{y}_{n}^{(f),\ast})=0,\forall f\in\mathcal{F},\forall n\in\mathcal{N}\setminus\text{Dst}(f)$. If $\alpha_{n}\geq\frac{1}{2}(d_{n}+1),\forall n\in\mathcal{N}$, where $d_{n}$ is the degree of node $n$, then the action $\mathbf{y}[t]=[x_{f}[t];\mu_{l}^{(f)}[t]]_{f\in\mathcal{F},l\in\mathcal{L}}$ jointly chosen in Algorithm 1 at each iteration $t\in\{0,1,\ldots\}$ satisfies

[TABLE]

where \Phi[t]=\sum_{f\in\mathcal{F},n\in\mathcal{N}}\big{(}\alpha_{n}\mathbf{1}_{\{n\neq\text{Dst}(f)\}}\|\mathbf{y}_{n}^{(f),\ast}-\mathbf{y}_{n}^{(f)}[t]\|^{2}+\alpha_{n}\mathbf{1}_{\{n=\text{Dst}(f)\}}\sum_{l\in\mathcal{I}(n)}(\mu_{l}^{(f),\ast}-\mu_{l}^{(f)}[t])^{2}\big{)}.

Proof:

See Appendix F. ∎

It remains to show that this modified backpressure algorithm leads to fundamentally improved performance.

IV-C Utility Optimality Gap Analysis

Define column vector \mathbf{Q}[t]=\big{[}Q_{n}^{(f)}[t]\big{]}_{f\in\mathcal{F},n\in\mathcal{N}\setminus\{\text{Dst}(f)\}} as the stacked vector of all virtual queues $Q_{n}^{(f)}[t]$ defined in (9). Note that (27) can be rewritten as $L(t)=\frac{1}{2}\|\mathbf{Q}[t]\|^{2}$. Define vectorized constraints (2) as $\mathbf{g}(\mathbf{y})=[g_{n}^{(f)}(\mathbf{y}_{n}^{(f)})]_{f\in\mathcal{F},n\in\mathcal{N}\setminus\text{Dst}(f)}$.

Lemma 8

Let $\mathbf{y}^{\ast}=[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$ be an optimal solution to problem (1)-(6) given in Fact 1, i.e., $g_{n}^{(f)}(\mathbf{y}_{n}^{(f),\ast})=0,\forall f\in\mathcal{F},\forall n\in\mathcal{N}\setminus\text{Dst}(f)$. If $\alpha_{n}\geq\frac{1}{2}(d_{n}+1),\forall n\in\mathcal{N}$ in Algorithm 1, where $d_{n}$ is the degree of node $n$, then for all $t\geq 1$,

[TABLE]

where \zeta=\Phi[-1]=\sum_{f\in\mathcal{F},n\in\mathcal{N}}\big{(}\alpha_{n}\mathbf{1}_{\{n\neq\text{Dst}(f)\}}\|\mathbf{y}_{n}^{(f),\ast}\|^{2}+\alpha_{n}\mathbf{1}_{\{n=\text{Dst}(f)\}}\sum_{l\in\mathcal{I}(n)}(\mu_{l}^{(f),\ast})^{2}\big{)} is a constant.

Proof:

By Lemma 7, we have $-\Delta[\tau]+f(\mathbf{y}[\tau])\geq f(\mathbf{y}^{\ast})+\Phi[t]-\Phi[t-1],\forall\tau\in\{0,1,\ldots,t-1\}$. Summing over $\tau\in\{0,1,\ldots,t-1\}$ yields

[TABLE]

where (a) follows from the fact that $\Phi[t]\geq 0,\forall t$.

Recall $\Delta[\tau]=L[\tau+1]-L[\tau]$, simplifying summations and rearranging terms yields

[TABLE]

where (a) follows from the fact that $L[0]=\mathbf{0}$ and $L[t]=\frac{1}{2}\|\mathbf{Q}[t]\|^{2}$. ∎

The next theorem summarizes that Algorithm 1 yields a vanishing utility optimality gap that approaches zero like $O(1/t)$.

Theorem 1

Let $\mathbf{y}^{\ast}=[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$ be an optimal solution to problem (1)-(6) given in Fact 1, i.e., $g_{n}^{(f)}(\mathbf{y}_{n}^{(f),\ast})=0,\forall f\in\mathcal{F},\forall n\in\mathcal{N}\setminus\text{Dst}(f)$. If $\alpha_{n}\geq\frac{1}{2}(d_{n}+1),\forall n\in\mathcal{N}$ in Algorithm 1, where $d_{n}$ is the degree of node $n$, then for all $t\geq 1$, we have

[TABLE]

where $\zeta$ is a constant defined in Lemma 8. Moreover, if we define $\overline{x}_{f}[t]=\frac{1}{t}\sum_{\tau=0}^{t-1}x_{f}[\tau],\forall f\in\mathcal{F}$, then

[TABLE]

Proof:

Recall that $f(\mathbf{y})=\sum_{f\in\mathcal{F}}U_{f}(x_{f})$. By Lemma 8, we have

[TABLE]

where (a) follows from the trivial fact that $\|\mathbf{Q}[t]\|^{2}\geq 0$.

Dividing both sides by a factor $t$ yields the first inequality in this theorem. The second inequality follows from the concavity of $U_{f}(\cdot)$ and Jensen’s inequality. ∎

IV-D Queue Stability Analysis

Lemma 9

Let $\mathbf{Q}[t],t\in\{0,1,\ldots\}$ be the virtual queues in Algorithm 1. For any $t\geq 1$,

[TABLE]

Proof:

This lemma follows directly from the fact that $\mathbf{Q}[0]=\mathbf{0}$ and queue update equation (9) can be written as $\mathbf{Q}[t+1]=\mathbf{Q}[t]+\mathbf{g}(\mathbf{y}[t-1])$. ∎

The next theorem shows the boundedness of all virtual queues $Q_{n}^{(f)}[t]$ in Algorithm 1.

Theorem 2

Let $\mathbf{y}^{\ast}=[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$ be an optimal solution to problem (1)-(6) given in Fact 1, i.e., $g_{n}^{(f)}(\mathbf{y}_{n}^{(f),\ast})=0,\forall f\in\mathcal{F},\forall n\in\mathcal{N}\setminus\text{Dst}(f)$, and $\boldsymbol{\lambda}^{\ast}$ be a Lagrange multiplier vector given in Assumption 2. If $\alpha_{n}\geq\frac{1}{2}(d_{n}+1)^{2},\forall n\in\mathcal{N}$ in Algorithm 1, where $d_{n}$ is the degree of node $n$, then for all $t\geq 1$,

[TABLE]

where $\zeta$ is a constant defined in Lemma 8.

Proof:

Let q(\boldsymbol{\lambda})=\sup_{\mathbf{y}\in\mathcal{C}}\big{\{}f(\mathbf{y})-\boldsymbol{\lambda}^{\mkern-1.5mu\mathsf{T}}\mathbf{g}(\mathbf{y})\big{\}} be the Lagrangian dual function defined in Assumption 2. For all $\tau\in\{0,1,\ldots,\}$, by Assumption 2, we have

[TABLE]

where $(a)$ follows from the definition of $q(\boldsymbol{\lambda}^{\ast})$. Rearranging terms yields

[TABLE]

Fix $t>0$. Summing over $\tau\in\{0,1,\ldots,t-1\}$ yields

[TABLE]

where (a) follows form Lemma 9 and (b) follows from Cauchy-Schwarz inequality.

On the other hand, by Lemma 8, we have

[TABLE]

Combining the last two inequalities and cancelling the common terms yields

[TABLE]

where (a) follows from the basic inequality $\sqrt{a+b}\leq\sqrt{a}+\sqrt{b}$ for any $a,b\geq 0$.

Thus, for any $f\in\mathcal{F}$ and $n\in\mathcal{N}\setminus\{\text{Dst}(f)\}$, we have

[TABLE]

∎

This theorem shows that the absolute values of all virtual queues $Q_{n}^{(f)}[t]$ are bounded by a constant $B=2\|\boldsymbol{\lambda}^{\ast}\|+\sqrt{2\zeta}$ from above. By Lemma 1 and discussions in Section III-A, the actual physical queues $Z_{n}^{(f)}[t]$ evolving via (8) satisfy $Z_{n}^{(f)}[t]\leq 2B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$. This is summarized in the next corollary.

Corollary 1

Let $\mathbf{y}^{\ast}=[x_{f}^{\ast};\mu_{l}^{(f),\ast}]_{f\in\mathcal{F},l\in\mathcal{L}}$ be an optimal solution to problem (1)-(6) given in Fact 1, i.e., $g_{n}^{(f)}(\mathbf{y}_{n}^{(f),\ast})=0,\forall f\in\mathcal{F},\forall n\in\mathcal{N}\setminus\text{Dst}(f)$, and $\boldsymbol{\lambda}^{\ast}$ be a Lagrange multiplier vector given in Assumption 2. If $\alpha_{n}\geq\frac{1}{2}(d_{n}+1)^{2},\forall n\in\mathcal{N}$ in Algorithm 1, where $d_{n}$ is the degree of node $n$, then all actual physical queues $Z_{n}^{(f)}[t],\forall f\in\mathcal{F},\forall n\in\mathcal{N}\setminus\{\text{Dst}(f)\}$ in the network evolving via (8) satisfy

[TABLE]

where $\zeta$ is a constant defined in Lemma 8.

IV-E Performance of Algorithm 1

Theorems 1 and 2 together imply that Algorithm 1 with $\alpha_{n}\geq\frac{1}{2}(d_{n}+1),\forall n\in\mathcal{N}$ can achieve a vanishing utility optimality gap that decays like $O(1/t)$, where $t$ is number of iterations, and guarantees the physical queues at each node for each session are always bounded by a constant that is independent of the utility optimality gap.

This is superior to existing backpressure algorithms from [5, 4, 10] that can achieve an $O(1/V)$ utility gap only at the cost of an $O(V^{2})$ or $O(V)$ queue length, where $V$ is an algorithm parameter. To obtain a vanishing utility gap, existing backpressure algorithms in [5, 4, 10] necessarily yield unbounded queues. To obtain a vanishing utility gap, existing backpressure algorithms in [5, 4] yield unbounded queues. We also comment that $O(V^{2})$ queue bound in the primal-dual type backpressure algorithm [5] is actually of the order $V^{2}\|\boldsymbol{\lambda}^{\ast}\|+B_{1}$ where $\boldsymbol{\lambda}^{\ast}$ is the Lagrangian multiplier vector attaining strong duality and $B_{1}$ is a constant determined by the problem parameters. A recent work [21] also shows that the $O(V)$ queue bound in the backpressure algorithm from drift-plus-penalty is of the order $V\|\boldsymbol{\lambda}^{\ast}\|+B_{2}$ where $B_{2}$ is also a constant determined by the problem parameters. Since $\boldsymbol{\lambda}^{\ast}$ is a constant vector independent of $V$, both algorithms are claimed to have $O(V^{2})$ or $O(V)$ queue bounds. By Corollary 1, Algorithm 1 guarantees physical queues at each node are bounded by $4\|\boldsymbol{\lambda}^{\ast}\|+B_{3}$, where $B_{3}$ is constant given a problem. Thus, the constant queue bound guaranteed by Algorithm 1 is typically smaller than the $O(V^{2})$ or $O(V)$ queue bounds from [5] and [21] even for a small $V$. (A small $V$ can yield a poor utility performance in the backpressure algorithms in [5, 4].)

V Numerical Experiment

In this section, we consider a simple network with $6$ nodes and $8$ links and $2$ sessions as described in Figure 1. This network has two sessions: session $1$ from node $1$ to node $6$ has utility function $\log(x_{1})$ and session $2$ from node $3$ to node $4$ has utility function $1.5\log(x_{2})$. (The log utilities are widely used as metrics of proportional fairness in the network [17].) The routing path of each session is arbitrary as long as data can be delivered from the source node to the destination node. For simplicity, assume that each link has capacity $1$. The optimal source session rate to problem (1)-(6) is $x^{\ast}_{1}=1.2$ and $x^{\ast}_{2}=1.8$ and link session rates, i.e., static routing for each session, is drawn in Figure 2.

To compare the convergence performance of Algorithm 1 and the backpressure algorithm in [4] (with the best utility-delay tradeoff among all existing backpressure algorithms), we run both Algorithm 1 with \alpha_{n}=\frac{1}{2}\big{(}d_{n}+1),\forall n\in\mathcal{N} and the backpressure algorithm in [4] with $V=500$ to plot Figure 3. It can be observed from Figure 3 that Algorithm 1 converges to the optimal source session rates faster than the backpressure algorithm in [4]. The backpressure algorithm in [4] with $V=400$ takes around $2500$ iterations to converges to source rates close to $(1.2,1.8)$ while Algorithm 1 only takes around $800$ iterations to converges to $(1.2,1.8)$ (as shown in the zoom-in subfigure at the top right corner.) In fact, the backpressure algorithm in [4] with $V=500$ can not converge to the exact optimal source session rate $(1.2,1.8)$ but can only converge to its neighborhood with a distance gap determined by the value of $V$. This is an effect from the fundamental $[O(1/V),O(V)]$ utility-delay tradeoff of the the backpressure algorithm in [4]. In contrast, Algorithm 1 can eventually converge to the the exact optimal source session rate $(1.2,1.8)$. A zoom-in subfigure at the bottom right corner in Figure 1 verifies this and shows that the source rate for Session $1$ in Algorithm 1 converges to $1.2$ while the source rate in the backpressure algorithm in [4] with $V=500$ oscillates around a point slightly larger than $1.2$.

Corollary 1 shows that Algorithm 1 guarantees each actual queue in the network is bounded by constant $4\|\boldsymbol{\lambda}^{\ast}\|+2\sqrt{2\xi}\|\mathbf{y}^{\ast}\|+\sum_{l\in\mathcal{O}(n)}C_{l}$. Recall that the backpressure algorithm in [4] can guarantee the actual queues in the network are bounded by a constant of order $V\|\boldsymbol{\lambda}^{\ast}\|$. Figure 4 plots the sum of actual queue length at each node for Algorithm 1 and the backpressure algorithm in [4] with $V=10,100$ and $500$. (Recall a larger $V$ in the backpressure algorithm in [4] yields a smaller utility gap but a larger queue length.) It can be observed that Algorithm 1 has the smallest actual queue length (see the zoom-in subfigure) and the actual queue length of the backpressure algorithm in [4] scales linearly with respect to $V$.

VI Conclusion

This paper develops a new first-order Lagrangian dual type backpressure algorithm for joint rate control and routing in multi-hop data networks. The new backpressure algorithm can achieve vanishing utility optimality gaps and finite queue lengths. This improves the state-of-art $[O(1/V),O(V^{2})]$ or $[O(1/V),O(V)]$ utility-delay tradeoff attained by existing backpressure algorithms [5, 9, 7, 10].

Appendix A Network Utility Maximization with Predetermined Multi-Path

Consider multi-path network utility maximization in [16] where each session has multiple given paths, then the source session rate $x_{f}$ in problem (1)-(6) becomes a vector $\mathbf{x}_{f}=[x_{f,j}]_{j\in\mathcal{P}_{f}}$ where $\mathcal{P}_{f}$ is the set of paths for session $f$ and the link session rate $\mu_{l}^{(f)}$ becomes a vector $\boldsymbol{\mu}_{l}^{(f)}=[\mu_{l}^{(f,j)}]_{j\in\mathcal{P}_{l}}$. Define $\mathcal{S}_{l}^{(f)}$ as the set of paths for session $f$ that are allowed to use link $l$. Note that if all paths of session $f$ are forbidden to use link $l$, then $\mathcal{S}_{l}^{(f)}=\emptyset$. The multi-path network utility maximization problem can be formulated as follows:

[TABLE]

The above formulation is in the form of problem (1)-(6) except that the variable dimension is extended.

Appendix B An Example Illustrating the Possibly Large Gap between Model (7) and Model (8)

Consider a network example shown in Figure 5. The network has $3k+1$ nodes where only node [math] is a destination; and $a_{i},i\in\{1,2,\ldots,k\}$ and $b_{i},i\in\{1,2,\ldots,k\}$ can have exogenous arrivals. Assume all link capacities are equal to 1; and the exogenous arrivals are periodic with period 2k, as follows:

• •

Time slot $1$: One packet arrives at node $a_{1}$.

• •

Time slot $2$: One packet arrives at node $a_{2}$.

• •

$\cdots$

• •

Time slot $k$: One packet arrives at node $a_{k}$.

• •

Time slot $k+1$: One packet arrives at node $b_{1}$.

• •

Time slot $k+2$: One packet arrives at node $b_{2}$.

• •

$\cdots$

• •

Time slot $2k$: One packet arrives at node $b_{k}$.

Under dynamics (7), each packet arrives on its own slot and traverses all links of its path to exit on the same slot it arrived. The queue backlog in each node is [math] for all time.

Under dynamics (8), the first packet arrives at time slot $1$ to node $a_{1}$. This packet visits node $a_{2}$ at time slot $2$, when the second packet also arrives at $a_{2}$. One of these packets is delivered to node $a_{3}$ at time slot $3$, and another packet also arrives to node $3$. The nodes $\{1,\ldots,k\}$ do not have any exogenous arrivals and act only to delay the delivery of all packets from the ai nodes. It follows that the link from node $k$ to node [math] will send exactly one packet over each slots $t\in\{2k+1,2k+2,\ldots,2k+k\}$. Similarly, the link from $b_{k}$ to [math] sends exactly one packet to node [math] over each of these same slots. Thus, node [math] receives $2$ packets on each slot $t\in\{2k+1,2k+2,\ldots,2k+k\}$, but can only output $1$ packet per slot. The queue backlog in this node grows linearly and reaches $k+1$ at time $2k+k$. Thus, the backlog in node [math] can be arbitrarily large when $k$ is large. This example demonstrates that, even when there is only one destination, the deviation between virtual queues under dynamics (7) and actual queues under dynamics (8) can be arbitrarily large, even when the in-degree and out-degree of $1$ and an in-degree of at most $2$.

Appendix C Proof of Part (2) in Lemma 1

Fix $f\in\mathcal{F},n\in\mathcal{N}\setminus\{\text{Dst}(f)\}$. By (10),

[TABLE]

where (a) follows from the fact that $\mu_{l}^{(f)}[t],x_{f}[t],\forall f,l,t$ are non-negative. Note that the right side of the above equation is identical to the right side of (7) and recall that $Y_{n}^{(f)}[0]=0<\widehat{Q}_{n}^{(f)}[0]$. By inductions, we have $Y_{n}^{(f)}[t]\leq\widehat{Q}_{n}^{(f)}[t],\forall t$. Since $\widehat{Q}_{n}^{(f)}[t]=Q_{n}^{(f)}[t]+B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$ and $Q_{n}^{(f)}[t]\leq B,\forall t$, we have $\widehat{Q}_{n}^{(f)}[t]\leq 2B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$. It follows that $Y_{n}^{(f)}[t]\leq 2B+\sum_{l\in\mathcal{O}(n)}C_{l},\forall t$.

Appendix D Proof of Lemma 3

Note that problem (18)-(20) satisfies Slater’s condition. So the optimal solution to problem (18)-(20) is characterized by KKT conditions [22]. Introducing Lagrange multipliers $\theta\in\mathbb{R}_{+}$ for inequality constraint $\sum_{k=1}^{K}z_{k}\leq b$ and $\boldsymbol{\nu}=[\nu_{1},\ldots,\nu_{K}]^{\mkern-1.5mu\mathsf{T}}\in\mathbb{R}_{+}^{K}$ for inequality constraints $z_{k}\geq 0,k\in\{1,2,\ldots,K\}$. Let $\mathbf{z}^{\ast}=[z_{1}^{\ast},\ldots,z_{K}^{\ast}]^{\mkern-1.5mu\mathsf{T}}$ and $(\theta^{\ast},\boldsymbol{\nu}^{\ast})$ be any primal and dual pair with the zero duality gap. By KKT conditions, we have z_{k}^{\ast}-a_{k}+\theta^{\ast}-\nu_{k}^{\ast}=0,\forall k\in\{1,2,\ldots,K\};\sum_{k=1}^{K}z_{k}^{\ast}\leq b;\theta^{\ast}\geq 0;\theta^{\ast}\big{(}\sum_{k=1}^{K}z_{k}^{\ast}-b\big{)}=0;z_{k}^{\ast}\geq 0,\forall k\in\{1,2,\ldots,K\};\nu_{k}^{\ast}\geq 0,\forall k\in\{1,2,\ldots,K\};\nu_{k}^{\ast}z_{k}^{\ast}=0,\forall k\in\{1,2,\ldots,K\}.

Eliminating $\nu_{k}^{\ast},\forall k\in\{1,2,\ldots,K\}$ in all equations yields \theta^{\ast}\geq a_{k}-z_{k}^{\ast},k\in\{1,2,\ldots,K\};\sum_{k=1}^{K}z_{k}^{\ast}\leq b;\theta^{\ast}\geq 0;\theta^{\ast}\big{(}\sum_{k=1}^{K}z_{k}^{\ast}-b\big{)}=0;z_{k}^{\ast}\geq 0,\forall k\in\{1,2,\ldots,K\};(z_{k}^{\ast}-a_{k}+\theta^{\ast})z_{k}^{\ast}=0,\forall k\in\{1,2,\ldots,K\}.

For all $k\in\{1,2,\ldots,K\}$, we consider $\theta^{\ast}<a_{k}$ and $\theta^{\ast}\geq a_{k}$ separately:

If $\theta^{\ast}<a_{k}$ , then $\theta^{\ast}\geq a_{k}-z_{k}^{\ast}$ holds only when $z_{k}^{\ast}>0$, which by $(z_{k}^{\ast}-a_{k}+\theta^{\ast})z_{k}^{\ast}=0$ implies that $z_{k}^{\ast}=a_{k}-\theta^{\ast}$. 2. 2.

If $\theta^{\ast}\geq a_{k}$, then $z_{k}^{\ast}>0$ is impossible, because $z_{k}^{\ast}>0$ implies that $z_{k}^{\ast}-a_{k}+\theta^{\ast}>0$, which together with $z_{k}^{\ast}>0$ contradicts the slackness condition $(z_{k}^{\ast}-a_{k}+\theta^{\ast})z_{k}^{\ast}=0$. Thus, if $\theta^{\ast}\geq a_{k}$, we must have $z_{k}^{\ast}=0$.

Summarizing both cases, we have $z_{k}^{\ast}=\max\{0,a_{k}-\theta^{\ast}\},\forall k\in\{1,2,\ldots,K\}$, where $\theta^{\ast}$ is chosen such that $\sum_{k=1}^{K}z_{k}^{\ast}\leq b$, $\theta^{\ast}\geq 0$ and \theta^{\ast}\big{(}\sum_{k=1}^{K}z_{k}^{\ast}-b\big{)}=0.

To find such $\theta^{\ast}$, we first check if $\theta^{\ast}=0$. If $\theta^{\ast}=0$ is true, the slackness condition \theta^{\ast}\big{(}\sum_{k=1}^{K}z_{k}^{\ast}-b\big{)} is guaranteed to hold and we need to further require $\sum_{k=1}^{K}z_{k}^{\ast}=\sum_{k=1}^{K}\max\{0,a_{k}\}\leq b$. Thus $\theta^{\ast}=0$ if and only if $\sum_{k=1}^{K}\max\{0,a_{k}\}\leq b$. Thus, Algorithm 2 check if $\sum_{k=1}^{K}\max\{0,a_{k}\}\leq b$ holds at the first step and if this is true, then we conclude $\theta^{\ast}=0$ and we are done!

Otherwise, we know $\theta^{\ast}>0$. By the slackness condition \theta^{\ast}\big{(}\sum_{k=1}^{K}z_{k}^{\ast}-b\big{)}=0, we must have $\sum_{k=1}^{K}z_{k}^{\ast}=\sum_{k=1}^{K}\max\{0,a_{k}-\theta^{\ast}\}=b$. To find $\theta^{\ast}>0$ such that $\sum_{k=1}^{K}\max\{0,a_{k}-\theta^{\ast}\}=b$, we could apply a bisection search by noting that all $z_{k}^{\ast}$ are decreasing with respect to $\theta^{\ast}$.

Another algorithm of finding $\theta^{\ast}$ is inspired by the observation that if $a_{j}\geq a_{i},\forall i,j\in\{1,2,\ldots,K\}$, then $z_{j}^{\ast}\geq z_{i}^{\ast}$. Thus, we first sort all $a_{k}$ in a decreasing order, say $\pi$ is the permutation such that $a_{\pi(1)}\geq a_{\pi(2)}\geq\cdots\geq a_{\pi(K)}$; and then sequentially check if $k\in\{1,2,\ldots,K\}$ is the index such that $a_{\pi(k)}-\theta^{\ast}\geq 0$ and $a_{\pi(k+1)}-\theta^{\ast}<0$. To check this, we first assume $k$ is indeed such an index and solve the equation $\sum_{j=1}^{k}(a_{\pi(j)}-\theta^{\ast})=b$ to obtain $\theta^{\ast}$; (Note that in Algorithm 2, to avoid recalculating the partial sum $\sum_{j=1}^{k}a_{\pi(j)}$ for each $k$, we introduce the parameter $S_{k}=\sum_{j=1}^{k}a_{\pi(j)}$ and update $S_{k}$ incrementally. By doing this, the complexity of each iteration in the loop is only $O(1)$.) then verify the assumption by checking if $\theta^{\ast}\geq 0$, $a_{\pi(k)}-\theta^{\ast}\geq 0$ and $a_{\pi(k+1)}-\theta^{\ast}\leq 0$. The algorithm is described in Algorithm 2 and has complexity $O(K\log(K))$. The overall complexity is dominated by the step of sorting all $a_{k}$.

Appendix E Proof of Lemma 6

The objective function (30) can be rewritten as

[TABLE]

where (a) follows from the fact that $g_{n}^{(f)}(\mathbf{y}_{n}^{(f)})=x_{f}\mathbf{1}_{\{n=\text{Src}(f)\}}+\sum_{l\in\mathcal{I}(n)}\mu_{l}^{(f)}-\sum_{l\in\mathcal{O}(n)}\mu_{l}^{(f)}$ and $\|\mathbf{y}_{n}^{(f)}-\mathbf{y}_{n}^{(f)}[t-1]\|^{2}=(x_{f}-x_{f}[t-1])^{2}\mathbf{1}_{\{n=\text{Src}(f)\}}+\sum_{l\in\mathcal{I}(n)}(\mu_{l}^{(f)}-\mu_{l}^{(f)}[t-1])^{2}+\sum_{l\in\mathcal{O}(n)}(\mu_{l}^{(f)}-\mu_{l}^{(f)}[t-1])^{2}$; and (b) follows by collecting each linear term $\mu_{l}^{(f)}$ and each quadratic term $(\mu_{l}^{(f)}-\mu_{l}^{(f)}[t-1])^{2}$. Note that each link session rate $\mu_{l}^{(f)}$ appears twice with opposite signs in the summation term \sum_{f\in\mathcal{F},n\in\mathcal{N}\setminus\{\text{Dst}(f)\}}W_{n}^{(f)}[t]\big{(}x_{f}\mathbf{1}_{\{n=\text{Src}(f)\}}+\sum_{l\in\mathcal{I}(n)}\mu_{l}^{(f)}-\sum_{l\in\mathcal{O}(n)}\mu_{l}^{(f)}\big{)} unless link $l$ flows into $\text{Dst}(f)$ and recall that $W_{\text{Dst}(f)}^{(f)}=0,\forall f\in\mathcal{F}$. The quadratic terms are collected in a similar way. Note that the term $\sum_{f\in\mathcal{F},n=\text{Dst}(f)}\alpha_{n}\sum_{l\in\mathcal{I}(n)}(\mu_{l}^{(f)}-\mu_{l}^{(f)}[t-1])^{2}$ introduced to the objective function (30) is necessary to guarantee each quadratic term $(\mu_{(m,n)}^{(f)}-\mu_{(m,n)}^{(f)}[t-1])^{2}$ with the same link index $(n,m)$ but different flow indices $f\in\mathcal{F}$ have the same coefficient $\alpha_{n}+\alpha_{m}$ in the last line of (32).

Note that equation (32) is now separable for each scalar $x_{f}$ and vector $[\mu_{(n,m)}^{(f)}]_{f\in\mathcal{F}}$. Thus, problem (30)-(31) can be decomposed into independent smaller optimization problems in the form of problem (12)-(13) with respect to each scalar $x_{f}$, and in the form of problem (14)-(17) with respect to each vector $[\mu_{(n,m)}^{(f)}]_{f\in\mathcal{F}}$.

Appendix F Proof of Lemma 7

Note that $W_{n}^{(f)}[t]$ appears as a known constant in (12). Since $U_{f}(x_{f})$ is concave and $W_{n}^{(f)}[t]x_{f}$ is linear, it follows that (12) is strongly concave with respect to $x_{f}$ with modulus $2\alpha_{n}$. Since $x_{f}[t]$ is chosen to solve (12)-(13), by Lemma 4, $\forall f\in\mathcal{F}$, we have

[TABLE]

Similarly, we know (14) is strongly concave with respect to vector $[\mu_{(n,m)}^{f}]_{f\in\mathcal{F}}$ with modulus $2(\alpha_{n}+\alpha_{m})$. By Lemma 4, $\forall(n,m)\in\mathcal{O}(n)$, we have

[TABLE]

Recall that each column vector $\mathbf{y}_{n}^{(f)}$ defined in (23) is composed by control actions that appear in each constraint (2); column vector $\mathbf{y}=[x_{f};\mu_{l}^{(f)}]_{f\in\mathcal{F},l\in\mathcal{L}}$ is the collection of all control actions; and $f(\mathbf{y})=\sum_{f\in\mathcal{F}}U_{f}(x_{f})$. Summing term (33)-I over all $f\in\mathcal{F}$ and term (34)-I over all $(n,m)\in\mathcal{L}$ and using an argument similar to the proof of Lemma 6 (Recall that $\mathbf{y}[t]$ is jointly chosen to minimize (30) by Lemma 6.) yields

[TABLE]

Recall that \Phi[t]=\sum_{f\in\mathcal{F},n\in\mathcal{N}}\big{(}\alpha_{n}\mathbf{1}_{\{n\neq\text{Dst}(f)\}}\|\mathbf{y}_{n}^{(f),\ast}-\mathbf{y}_{n}^{(f)}[t]\|^{2}+\alpha_{n}\mathbf{1}_{\{n=\text{Dst}(f)\}}\sum_{l\in\mathcal{I}(n)}(\mu_{l}^{(f),\ast}-\mu_{l}^{(f)}[t])^{2}\big{)}. Summing term (33)-II over all $f\in\mathcal{F}$ and term (34)-II over all $(n,m)\in\mathcal{L}$ yields

[TABLE]

Combining (33)-(36) and rearranging terms yields

[TABLE]

where (a) follows because $g_{n}^{(f)}(\mathbf{y}_{n}^{(f),\ast})=0,\forall f\in\mathcal{F},\forall n\in\mathcal{N}\setminus\text{Dst}(f),$ and $\sum_{f\in\mathcal{F},n=\text{Dst}(f)}\alpha_{n}\sum_{l\in\mathcal{I}(n)}(\mu_{l}^{(f)}[t]-\mu_{l}^{(f)}[t-1])^{2}\geq 0$; (b) follows from the fact that $W_{n}^{(f)}[t]=Q_{n}^{(f)}[t]+g_{n}^{(f)}(\mathbf{y}_{n}^{(f)}[t-1])$.

Recall that $u_{1}^{\mkern-1.5mu\mathsf{T}}u_{2}=\frac{1}{2}u_{1}^{2}+\frac{1}{2}u_{2}^{2}-\frac{1}{2}(u_{1}-u_{2})^{2}$ for any $u_{1},u_{2}\in\mathbb{R}$. Thus, for all $f\in\mathcal{F},n\in\mathcal{N}\setminus\text{Dst}(f)$, we have

[TABLE]

Substituting (38) into (37) yields

[TABLE]

where (a) follows from the Fact 2, i.e., each $g_{n}^{(f)}(\cdot)$ is Lipschitz with modulus $\beta_{n}$ and (b) follows because $\alpha_{n}\geq\frac{1}{2}(d_{n}+1)$, $\beta_{n}\leq\sqrt{d_{n}+1}$ and \frac{1}{2}\big{(}g_{n}^{(f)}(\mathbf{y}_{n}^{(f)}[t-1])\big{)}^{2}\geq 0.

Subtracting (28) from (39) and cancelling the common terms on both sides yields

[TABLE]