Proportional Fair RAT Aggregation in HetNets

TL;DR

This paper proposes a simple distributed algorithm for resource allocation in multi-RAT HetNets that achieves proportional fairness, ensuring improved throughput and fairness without requiring inter-BS coordination.

Contribution

It introduces a novel distributed resource allocation algorithm for multi-RAT HetNets that guarantees convergence to proportional fairness without inter-BS coordination.

Findings

The algorithm converges to the proportional fairness solution.

It provides bounds on convergence speed.

The algorithm's outcomes are proven to be optimal.

Abstract

Heterogeneity in wireless network architectures (i.e., the coexistence of 3G, LTE, 5G, WiFi, etc.) has become a key component of current and future generation cellular networks. Simultaneous aggregation of each client's traffic across multiple such radio access technologies (RATs) / base stations (BSs) can significantly increase the system throughput, and has become an important feature of cellular standards on multi-RAT integration. Distributed algorithms that can realize the full potential of this aggregation are thus of great importance to operators. In this paper, we study the problem of resource allocation for multi-RAT traffic aggregation in HetNets (heterogeneous networks). Our goal is to ensure that the resources at each BS are allocated so that the aggregate throughput achieved by each client across its RATs satisfies a proportional fairness (PF) criterion. In particular, we provide a simple distributed algorithm for resource allocation at each BS that extends the PF allocation algorithm for a single BS. Despite its simplicity and lack of coordination across the BSs, we show that our algorithm converges to the desired PF solution and provide (tight) bounds on its convergence speed. We also study the characteristics of the optimal solution and use its properties to prove the optimality of our algorithm's outcomes.

Tables

Table 1. TABLE I: Main Notation

$𝐍$ and $N$: Set and number of all clients in the network

$𝐌$ and $M$: Set and number of all BSs in the network

$R_{i,j}$: PHY rate of client $i$ to BS $j$

$R_{max}$: maximum PHY rate across all clients and BSs

$R_{min}$: non-zero minimum PHY rate across all clients and BSs

${\lambda }_{i,j}$: Fraction of time allocated to client $i$ by BS $j$

$𝝀$: Vector of ${\lambda }_{i,j}$s across all clients and BSs

$r_i$: Total throughput of client $i$ across all its RATs

${\omega }_i$: A positive number that represents client $i$’s weight or priority

${\theta }_j$: Water-fill level at BS $j$

Equations

$$\text{Total Throughput of Client }i=r_i=\sum _{j=1}^M{\lambda }_{i,j}{R}_{i,j}$$

$$\sum _{i=1}^N{\lambda }_{i,j}\le 1\forall j\in \mathbf{M}$$

$${\lambda }_{i,j}\ge 0\forall i\in \mathbf{N},\mathbf{j}\in \mathbf{M}$$

$${\mathcal{P}}_1:\max$$

$$L(\mathbf{\lambda },\mu )=\sum _{i=1}^{n^{\prime }}{\omega }_i\log (R_{i,j}{\lambda }_i)+\mu (1-\sum _{i=1}^{n^{\prime }}{\lambda }_i)$$

$$\frac{R_{i,j}{\omega }_i}{R_{i,j}{\lambda }_i}-\mu =0⟹\frac{{\omega }_i}{{\lambda }_i}=\mu \forall i\in \{1,...,n^{\prime }\}$$

$$\frac{{\lambda }_i}{{\omega }_i}={\theta }_j\forall i\in \{1,...,n^{\prime }\}⟹\boxed{\frac{r_i}{{\omega }_i{R}_{i,j}}={\theta }_j\forall i\in \{1,...,n^{\prime }\}}$$

$$\frac{r_1^{\prime }+{\lambda }_{1,j}{R}_{1,j}}{{\omega }_1{R}_{1,j}}=\frac{r_2^{\prime }+{\lambda }_{2,j}{R}_{2,j}}{{\omega }_2{R}_{2,j}}=...=\frac{r_k^{\prime }+{\lambda }_{k,j}{R}_{k,j}}{{\omega }_k{R}_{k,j}}={\theta }_j$$

$$\frac{r_k^{\prime }}{{\omega }_k{R}_{k,j}}<{\theta }_j\le \frac{r_{k+1}^{\prime }}{{\omega }_{k+1}{R}_{k+1,j}}$$

$$\sum _{i=1}^k{\lambda }_{i,j}=1,{\lambda }_{i,j}>0$$

$$\{\begin{array}{l}\frac{\frac{r_2^{\prime }{\omega }_1{R}_{1,j}}{{\omega }_2{R}_{2,j}}-r_1^{\prime }}{R_{1,j}}\ge 1\Rightarrow k=1\text{else}\\ \frac{\frac{r_3^{\prime }{\omega }_1{R}_{1,j}}{{\omega }_3{R}_{3,j}}-r_1^{\prime }}{R_{1,j}}+\frac{\frac{r_3^{\prime }{\omega }_2{R}_{2,j}}{{\omega }_3{R}_{3,j}}-r_2^{\prime }}{R_{2,j}}\ge 1\Rightarrow k=2\text{else}\\ ...\\ \frac{\frac{r_{n^{\prime }}^{\prime }{\omega }_1{R}_{1,j}}{{\omega }_{n^{\prime }}{R}_{n^{\prime },j}}-r_1^{\prime }}{R_{1,j}}+...+\frac{\frac{r_{n^{\prime }}^{\prime }{\omega }_{n^{\prime }-1}{R}_{n^{\prime }-1,j}}{{\omega }_{n^{\prime }}{R}_{n^{\prime },j}}-r_{n^{\prime }-1}^{\prime }}{R_{n^{\prime }-1,j}}\ge 1\\ \Rightarrow k=n^{\prime }-1\text{else}\\ k=n^{\prime }\end{array}$$

$$\sum _{i=1}^k\frac{{\theta }_j{\omega }_i{R}_{i,j}-r_i^{\prime }}{R_{i,j}}=1$$

$$\frac{r_i}{{\omega }_i{R}_{i,j}}<\frac{r_{i^{\prime }}}{{\omega }_{i^{\prime }}{R}_{i^{\prime },j}}$$

$$\begin{array}{r}f(\mathbf{\lambda }{)}^{new}-f(\mathbf{\lambda }{)}^{old}={\omega }_i\log (r_i+\delta R_{i,j})-\\ {\omega }_i\log (r_i)+{\omega }_{i^{\prime }}\log (r_{i^{\prime }}-\delta R_{i^{\prime },j})-{\omega }_{i^{\prime }}\log (r_{i^{\prime }})=\\ {\omega }_i\log (1+\delta \frac{R_{i,j}}{r_i})+{\omega }_{i^{\prime }}\log (1-\delta \frac{R_{i^{\prime },j}}{r_{i^{\prime }}})\end{array}$$

$$g(\delta )={\omega }_i\log (1+\delta \frac{R_{i,j}}{r_i})+{\omega }_{i^{\prime }}\log (1-\delta \frac{R_{i^{\prime },j}}{r_{i^{\prime }}})$$

$$\begin{array}{r}{g}^{\prime }(\delta )={\omega }_i\frac{\frac{R_{i,j}}{r_i}}{1+\delta \frac{R_{i,j}}{r_i}}-{\omega }_{i^{\prime }}\frac{\frac{R_{i^{\prime },j}}{r_{i^{\prime }}}}{1-\delta \frac{R_{i^{\prime },j}}{r_{i^{\prime }}}}\\ =\frac{{\omega }_i{R}_{i,j}}{r_i+\delta R_{i,j}}-\frac{{\omega }_{i^{\prime }}{R}_{i^{\prime },j}}{r_{i^{\prime }}-\delta R_{i^{\prime },j}}=\frac{{\omega }_i{R}_{i,j}}{r_i^{new}}-\frac{{\omega }_{i^{\prime }}{R}_{i^{\prime },j}}{r_{i^{\prime }}^{new}}\ge 0\end{array}$$

$$\begin{array}{r}g(\delta )\stackrel{\text{Taylor Approx}}{\approx }{\omega }_i\delta \frac{R_{i,j}}{r_i}-{\omega }_{i^{\prime }}\delta \frac{R_{i^{\prime },j}}{r_{i^{\prime }}}=\\ \delta (\frac{{\omega }_i{R}_{i,j}}{r_i}-\frac{{\omega }_{i^{\prime }}{R}_{i^{\prime },j}}{r_{i^{\prime }}})>0\end{array}$$

$$\frac{r_{i_1}^{old}}{{\omega }_{i_1}{R}_{i_1,j}}\le \frac{r_{i_2}^{old}}{{\omega }_{i_2}{R}_{i_2,j}}\le ...\le \frac{r_{i_q}^{old}}{{\omega }_{i_q}{R}_{i_q,j}}$$

$$r_{i_1}^{new}=r_{i_1},r_{i_1}^{old}=r_{i_1}-{ϵ}_{i_1}{R}_{i_1,j}$$

$$r_{i_p}^{new}=r_{i_p},r_{i_p}^{old}=r_{i_p}+{ϵ}_{i_p}{R}_{i_p,j}\forall i_p\in \{i_2,...,i_q\}$$

$${ϵ}_{i_1}={ϵ}_{i_2}+{ϵ}_{i_3}+...+{ϵ}_{i_q}$$

$$\frac{r_{i_1}}{{\omega }_{i_1}{R}_{i_1,j}}\le \frac{r_{i_p}}{{\omega }_{i_p}{R}_{i_p,j}}\forall i_p\in \{i_2,...,i_q\}$$

$$⟹\frac{R_{i_p,j}}{r_{i_p}}\le \frac{{\omega }_{i_1}}{{\omega }_{i_p}}\frac{R_{i_1,j}}{r_{i_1}}$$

$$\begin{array}{r}f(\mathbf{\lambda }{)}^{old}-f(\mathbf{\lambda }{)}^{new}=\\ {\omega }_{i_1}\log (1-\frac{{ϵ}_{i_1}{R}_{i_1,j}}{r_{i_1}})+\sum _{p=2}^q{\omega }_{i_p}\log (1+\frac{{ϵ}_{i_p}{R}_{i_p,j}}{r_{i_p}})\stackrel{\text{Eq. (refeqct:5)}}{\le }\\ {\omega }_{i_1}\log (1-\frac{{ϵ}_{i_1}{R}_{i_1,j}}{r_{i_1}})+\sum _{p=2}^q{\omega }_{i_p}\log (1+\frac{{ϵ}_{i_p}{R}_{i_1,j}}{r_{i_1}}\frac{{\omega }_{i_1}}{{\omega }_{i_p}})\end{array}$$

$$\begin{array}{r}\sum _{p=2}^q{\omega }_{i_p}\log (1+x_p)=W\sum _{p=2}^q\frac{{\omega }_{i_p}}{W}\log (1+x_p)\le \\ W\log (\sum _{p=2}^q(\frac{{\omega }_{i_p}}{W}+\frac{{\omega }_{i_p}}{W}{x}_p))=W\log (1+\sum _{p=2}^q\frac{{\omega }_{i_p}}{W}{x}_p)\end{array}$$

$$\begin{array}{r}\le {\omega }_{i_1}\log (1-\frac{{ϵ}_{i_1}{R}_{i_1,j}}{r_{i_1}})+W\log (1+\frac{{\omega }_{i_1}}{W}\frac{R_{i_1,j}}{r_{i_1}}\stackrel{={ϵ}_{i_1}}{\overbrace{\sum _{p=2}^q{ϵ}_{i_p}}})=\\ {\omega }_{i_1}[\log (1-z)+\gamma \log (1+\frac{z}{\gamma })]\stackrel{\text{Taylor Series}}{\le }-{\omega }_{i_1}\frac{z^2}{2}\\ ⟹f(\mathbf{\lambda }{)}^{new}-f(\mathbf{\lambda }{)}^{old}\ge {\omega }_{i_1}\frac{z^2}{2}\end{array}$$

$$\text{Convergence Time}\le \frac{\text{Max}f(\mathbf{\lambda })-\text{Min}f(\mathbf{\lambda })}{\frac{1}{2}{\omega }_{min}{ϵ}^2(\frac{R_{i_1,j}}{r_{i_1}}{)}^2}$$

$$\le \frac{({\sum }_{i=1}^N{\omega }_i)(\log (r_{max})-\log (r_{min}))}{\frac{1}{2}{\omega }_{min}{ϵ}^2(\frac{R_{min}}{M{R}_{max}}{)}^2}\le$$

$$\frac{(\sum {\omega }_i)(\log (M{R}_{max})-\log (\frac{{\omega }_{min}}{\sum {\omega }_i}{R}_{min}))}{\frac{1}{2}{\omega }_{min}{ϵ}^2(\frac{R_{min}}{M{R}_{max}}{)}^2}$$

$$\equiv O(\frac{N{M}^2\log (MN)}{{ϵ}^2})$$

$$R_{i,j},{\lambda }_{i,j}^{eq}>0⟹\frac{r_i^{eq}}{{\omega }_i{R}_{i,j}}={\theta }_j^{eq}$$

$$R_{i,j}>0,{\lambda }_{i,j}^{eq}=0⟹\frac{r_i^{eq}}{{\omega }_i{R}_{i,j}}\ge {\theta }_j^{eq}$$

$$\sum _{i=1}^N{\omega }_i=\sum _{i=1}^N\frac{{\omega }_i{r}_i^{eq}}{r_i^{eq}}=\sum _{i=1}^N\sum _{j=1}^M\frac{{\omega }_i{\lambda }_{i,j}^{eq}{R}_{i,j}}{r_i^{eq}}=\sum _{{\lambda }_{i,j}^{eq}>0}\frac{{\omega }_i{\lambda }_{i,j}^{eq}{R}_{i,j}}{r_i^{eq}}$$

Full text

I Introduction

The increasing demand for wireless data has led to denser and more heterogeneous wireless network deployments. This heterogeneity manifests itself in terms of network deployments across multiple radio access technologies (e.g., 3G, LTE, WiFi, 5G), cell sizes (e.g., macro, pico, femto), and frequency bands (e.g., TV bands, 1.8-2.4 GHz, mmWave), $etc$. To realize the gains associated with such heterogeneous networks (HetNets), consumer (client) devices are also being equipped with an increasing number of radio access technologies (RATs), and some are already able to simultaneously aggregate the traffic across multiple RATs to increase throughput [1].

To support such traffic aggregation on the network side, the 3GPP (3rd generation partnership project) has been actively developing multi-RAT integration solutions. The introduction of LWA (LTE-WiFi Aggregation) as part of the 3GPP Release 13 [2] was a step in this direction. LWA allows using both LTE and WiFi links for a single traffic flow and is generally more efficient than transport layer aggregation protocols (e.g., MultiPath TCP), due to coordination at lower protocol stack layers. LWA’s design primarily follows the LTE Dual Connectivity (DC) architecture (defined in 3GPP Release 12 [3]), which allows a wireless device to connect to two LTE eNBs that are on different carrier frequencies, and utilize the radio resources that belong to both of them. Currently, the 3GPP is working on a solution to support below IP (layer 2) multi-RAT integration across any combination of RATs, including LTE, WiFi, 802.11ad/ay, and 5G New Radio (NR) [4]. The proposed architecture would allow for dynamic traffic splitting across RATs for each client, which can lead to a significant increase in the system performance (e.g., total throughput).

However, it is difficult to design resource allocation algorithms for each BS111We use “BS” generically to mean an LTE eNB, WiFi AP, etc. that realize the performance benefits of such integrated HetNets. Specifically, (i) backhaul links from different BSs in HetNets show diverse capacity and latency characteristics and depend on the underlying backhauling technology. For example, cable and DSL have on average 28 and 62 ms roundtrip latencies, respectively [5, 6]. The latency can be even higher when a network operator uses a third party ISP to communicate with its BSs (e.g., a mobile operator that uses a wired ISP to control its WiFi BSs). Such latencies make it infeasible for BSs to communicate with each other or a central controller for real-time resource allocation at each BS. As a result, any practical resource allocation algorithm for multi-RAT HetNets should be fully distributed (i.e., autonomously executed by each BS). (ii) Resource allocation has many practical constraints. Conventional BS hardware allows only minor modifications to existing resource allocation algorithms through software updates, limiting the algorithm design space. New algorithms should also incur minimal signaling overhead and computational complexity. Distributed algorithms based on the traditional network utility maximization framework [7, 8] do not meet these requirements, because as we will show later through simulations the resulting algorithms are radically different from how conventional BSs operate, have significant over-the-air signaling overhead, and increase the computational complexity on the client side. (iii) In HetNets, each client has access to a client-specific set of RATs, and receives packets at a different PHY rate on each RAT. These rates are naturally different across clients. This multi-rate property of HetNets makes it particularly challenging to design resource allocation algorithms with performance guarantee. As a result, existing solutions in the literature are all limited to simple setups, e.g., when each client has only two RATs as in the case of LWA [9] or LTE DC [10].

In this paper, we study the problem of resource allocation for traffic aggregation in multi-RAT HetNets. We focus on the proportional-fair (PF) fairness objective as it is widely used and implemented in BSs and provides a balance between fairness and throughput [11, 12]. We first consider PF resource allocation in a single BS, and then use our insights from this case to design a distributed algorithm that meets our three research challenges. We next show that our algorithm converges to an optimal PF resource allocation. The key contributions are as follows:

• •

Algorithm Design: We study the basics of PF resource allocation in a single BS to gain intuition for the distributed algorithm design. We show that PF resource allocation in a single BS can be viewed as a special type of water-filling. We generalize this observation to a new fully distributed water-filling algorithm (named AFRA) that makes a minor modification to the conventional single BS algorithm and achieves PF in HetNets.

• •

Convergence and Speed: We show that AFRA is guaranteed to converge to an equilibrium as BSs autonomously execute it [Theorem 1] and derive tight bounds on its convergence time (speed) [Theorem 2].

• •

Optimality: We first show that at optimality, the sum of the inverse water-fill levels across all BSs is equal to the sum of the weights (numbers that show clients’ priorities) across all clients [Theorem 3]. Next, we use this property to prove that any equilibrium outcome of AFRA is globally optimal [Theorem 4]. Finally, we show that at equilibrium the vector of throughput rates across all clients is unique; however, there could be infinitely many resource allocations that realize this outcome [Theorem 5].

• •

Practicality: We construct a testbed with programmable BS hardware, and show that we can successfully aggregate the throughput across multiple BSs at the MAC layer. We also show that replacing the conventional resource allocation algorithm on each BS with AFRA can substantially increase the system throughput and fairness.

• •

Performance: We conduct extensive simulations to characterize AFRA’s convergence time properties as we scale the number of BSs and clients. We also introduce policies that reduce the convergence time by more than 30%. Finally, we compare the performance of AFRA against DDNUM, a dual decomposition algorithm that we derived from the NUM framework. We show that compared to DDNUM, AFRA is 2-3 times faster with 4-5 times less over-the-air overhead.

This paper is organized as follows. We discuss the related work in Section II. We present the system model and details of AFRA in Section III. In Sections IV and V we prove the convergence and optimality of AFRA. We present the results of our experiments, simulations, and comparisons against DDNUM in Section 4. We conclude the paper in Section VII.

II Related Work

We discuss the related work in the areas of multi-BS communication and distributed optimization, and highlight their differences from this paper.

Single-RAT Multi-BS Communication. Prior works have studied the problem of traffic aggregation when a client can simultaneously communicate with multiple same technology BSs. For example, [13] uses game theory to model selfish traffic splitting by each client in WLANs. On the other hand, the resource allocation problem in HetNets is primarily addressed at the BS side. Similarly, [10] proposes an approximation algorithm to address the problem of client association and traffic splitting in LTE DC. Our algorithm (AFRA) goes beyond this and other related work by guaranteeing optimal resource allocation for any number of RATs and BSs. Other works have developed centralized client association algorithms to achieve max-min [14] and proportional fairness [15] in multi-rate WLANs. In contrast, the problem of resource allocation in HetNets needs to be solved in a fully distributed manner.

Multi-RAT Communication. Resource allocation algorithms that realize the capacity gains in HetNets are still in their early stages. The problem of PF resource allocation for LWA was studied in [9]. In the proposed setup, each client has one LTE and one WiFi RAT. Further, there is only a single LTE BS in the network, and each client’s throughput across its WiFi RAT is fixed. Next, the authors propose a water-filling based resource allocation algorithm at the LTE BS that achieves PF. Similarly, we show that the optimal PF resource allocation in a single BS can be interpreted as a form of water-filling. However, we use the observation to design an optimal algorithm for the generic problem with any number of BSs and client RATs, and explicitly model the impact of system dynamics on the throughput that each client gets from every BS. In our prior work [16], we addressed the problem of max-min fair resource allocation in HetNets. However, even with opportunistic centralized network supervision over autonomous resource allocation at each BS we could not optimally solve the problem. Here, we focus on the PF objective, which is commonly implemented in BSs, and show that we can optimally solve the problem in a purely distributed manner. Other works have built testbeds to evaluate the over-the-air performance of MAC-level cross-RAT throughput aggregation [17, 18, 19, 20]. All these works have relied on conventional scheduling algorithms on each BS and focused on higher layer transport and application performance. We experimentally show that replacing the conventional resource allocation algorithms with AFRA can substantially increase the system throughput and fairness.

Distributed Network Utility Maximization (NUM). There is a large body of general results on the mathematics of distributed computation, some of which are summarized in standard textbooks such as [21, 22]. More recently, the framework of NUM [7, 8, 23] has emerged as a mathematical tool to optimize layered network architectures. The framework allows for decomposition of a global optimization problem into subsets of local problems that are carried out distributedly and implicitly solve the global NUM problem. We have derived an alternative distributed algorithm (named DDNUM) by leveraging dual decomposition and the NUM framework. We will show through simulations that DDNUM is 2-3 times slower than AFRA (in terms of convergence time) and increases the over-the-air signaling overhead by 4-5 times. These disadvantages, coupled with the increased client side computational complexity and lack of compatibility with conventional BSs, make NUM-based algorithms impractical for multi-RAT traffic aggregation.

III System Model

We discuss the system model and the resource allocation algorithm that is autonomously executed by each BS.

III-A Network Model

We consider a HetNet composed of a set of BSs $\bf M$ = $\{1,...,M\}$ and a set of clients $\bf N$ = $\{1,...,N\}$. Each BS has a limited transmission range and can only serve clients within its range. Each client has a client-specific number of RATs, and therefore has access to a subset of BSs. We model clients that can aggregate traffic across BSs of the same technology (e.g., LTE DC) with multiple such RATs. Fig 1 shows an example HetNet topology. We assume that clients split their traffic over the BSs and focus on the resource allocation problem at each BS. It is itself a challenging problem to determine which BS to associate with among same technology BSs (e.g., choosing the optimal LTE BS if a client has an LTE RAT). We assume there exists a rule to pre-determine client RAT to BS association. The pre-determination rule could for instance be any load balancing algorithm [24, 25], or based on the received signal strength. Similar to [13, 14, 15, 16, 24], we assume that the transmission in one BS does not interfere with an adjacent BS. This can be achieved through spectrum separation between BSs that belong to different access networks and frequency reuse among same technology BSs.

III-B Throughput Model

We consider a multi-rate system and use $R_{i,j}$ to denote the PHY rate of client $i$ to BS $j$. Since each BS generally serves more than one client, clients of the same BS need to share resources such as time and frequency slots (e.g. in 3/4/5G) or transmission opportunities (e.g. in WiFi). The throughput achieved by client $i$ from BS $j$ thus depends on the load of the BS and will be a fraction of $R_{i,j}$. We assume that each BS employs a TDMA throughput sharing model222In Section VI-A, we discuss how we can extend our model and algorithm to capture practical implementation issues such as WiFi contention. and let $\lambda_{i,j}$ denote the fraction of time allocated to client $i$ by BS $j$. Hence, the throughput achieved by client $i$ from BS $j$ is equal to $\lambda_{i,j}R_{i,j}$ and its total throughput across all its RATs would be

[TABLE]

The total amount of time fractions available to each BS cannot exceed 1. Thus, for the $\lambda_{i,j}s$ to be feasible we have

[TABLE]

[TABLE]

III-C Background: Conventional PF Allocation in a Single BS

We first describe the basics of the PF resource allocation that is conventionally implemented in today’s BSs. Consider a network topology consisting of only a single BS j and $n^{\prime}$ clients. Let $r_{i}$ denote the throughput of client $i$ and $\omega_{i}$ a positive number that denotes its weight (or priority). A widely used objective function for PF is to maximize $\sum_{i=1}^{n^{\prime}}\omega_{i}\log(r_{i})$ [11, 12]. It represents a tradeoff between throughput and fairness among the clients. Let $\lambda_{i}$ denote the time fraction allocated to client $i$ by BS $j$. To maximize the PF objective function, the BS needs to solve the following problem

[TABLE]

Problem $\mathcal{P}_{1}$ can be easily solved through a simple algorithm. The Lagrangian of $\mathcal{P}_{1}$ can be expressed as

[TABLE]

where $\mu$ is a constant number (Lagrange multiplier) chosen to meet the time resource constraint. Differentiating with respect to time fraction resource $\lambda_{i}$ and setting to zero gives

[TABLE]

Since the sum of time fractions at optimality is equal to 1, we can conclude from Eq. (5) that $\mu={\sum\omega_{i}}$. With known $\mu$ and $\omega_{i}$, we can derive $\lambda_{i}$s from Eq. (5).

Now, let ${\theta}_{j}$ be defined as $\frac{1}{\mu}$. Leveraging Eq. (5), we have

[TABLE]

Eq. (6) has an interesting water-filling based interpretation: the time allocated to each client is such that the throughput of the client divided by its PHY rate times its weight is the same across all clients. We refer to this ratio (i.e., ${\theta}_{j}$) as the water-fill level of BS $j$. In the next section, we will turn this observation in a single BS into a distributed resource allocation algorithm in HetNets.

III-D Distributed Resource Allocation in HetNets

There are two approaches to designing a resource allocation algorithm for generic HetNets. One approach, as we show in the Appendix, is to extend the formulation in $\mathcal{P}_{1}$ to include multiple BSs and client RATs, and use dual decomposition to derive a distributed algorithm. This approach converges to the optimal solution; however, the Lagrange multipliers across BSs would no longer correspond to BSs’ water-fill levels. The second approach is to directly generalize the water-filling interpretation to derive an alternative algorithm, which still converges to the optimal solution (Section V) with far less overhead, convergence time, and complexity than the dual decomposition based algorithm (Section VI-C).

From Eq. (6), we observe that in a network with only a single BS, the BS allocates its time resources so that the clients who get the time resources reach the same water-fill level (i.e., throughput divided by $\omega_{i}R_{i,j}$). Thus, in generic HetNets, if each BS considers the total throughput of each client across all its RATs (i.e., $r_{i}$) divided by $\omega_{i}R_{i,j}$ in its water-fill definition, this should lead to a fair distributed algorithm. In other words, each BS $j$ should share its time resources across its clients such that: (1) all clients who get the time resources reach the same water-fill level at BS $j$ (i.e., ${\theta}_{j}$), and (2) if a client (e.g., $i^{\prime}$) does not get any time resources from BS $j$, its $\frac{r_{i^{\prime}}}{\omega_{i^{\prime}}R_{i^{\prime},j}}$ is greater than ${\theta}_{j}$. Fig. 2 illustrates this operation.

We next turn this idea into a distributed resource allocation algorithm. Consider slotted time for now. Algorithm AFRA (Fig. 3) summarizes the steps that are autonomously executed by each BS $j$. There are three main steps in the algorithm: (i) clients are sorted based on the total throughout they receive from other BSs ($r^{\prime}_{i}$) divided by $\omega_{i}R_{i,j}$ (Line 3), (ii) BS $j$ finds the water-fill level (${\theta}_{j}$) and allocates the time resources accordingly (Line 4), and (iii) finally we introduce a randomization parameter to limit concurrent resource adaptation of a single client by multiple BSs (Line 5).

We next elaborate on how each BS $j$ finds its water-fill level and its clients’ time resource fractions (Line 4). Let $n^{\prime}$ denote the number of clients such that $R_{i,j}>0$. Let $r^{\prime}_{i}$ denote the total throughput of client $i$ from all BSs other than $j$. Consider an ordering in clients’ $\frac{r^{\prime}_{i}}{\omega_{i}R_{i,j}}$ according to Line 3 of AFRA. In order to solve the water-fill problem (i.e., Line 4 of AFRA), we need to find the water-fill level ${\theta}_{j}$, client index $k$, and time fractions $\lambda_{i,j}$s such that

[TABLE]

We can find these variables with a simple set of linear operations. First, we can find $k$ by checking a set of inequalities

[TABLE]

In the first inequality, we first check if $\frac{r^{\prime}_{1}+R_{1,j}}{\omega_{1}R_{1,j}}\leq\frac{r^{\prime}_{2}}{\omega_{2}R_{2,j}}$. If this is true, from Eq. (7) we conclude that client 2 would have a higher $\frac{r^{\prime}_{2}}{\omega_{2}R_{2,j}}$ than $\frac{r_{1}}{\omega_{1}R_{1,j}}$ even if BS $j$ allocated all its time resources to client 1 (i.e., to the client with minimum $\frac{r^{\prime}_{i}}{\omega_{i}R_{i,j}}$ across all $n^{\prime}$ clients). As a result $k$ should be equal to 1. This procedure (and logic) is continued until $k$ is found.

With known $k$, we can find ${\theta}_{j}$ by combining Eqs. (7) and (9) and solving the following linear equation

[TABLE]

With known $k$ and ${\theta}_{j}$, the $\lambda_{i,j}$s can be found from Eq. (7).

AFRA’s Computational Complexity and Message Passing Overhead. We calculate AFRA’s computational complexity in finding the new time resource fractions ($\lambda_{i,j}$s) for a BS $j$. Let $n^{\prime}$ denote the number of clients with non-zero PHY rates to $j$. The complexity of sorting clients (Line 3) is $O(n^{\prime}\log(n^{\prime}))$. The complexity of finding the water-fill level and the new time resource fractions (Line 4) is $O(n^{\prime}\log(n^{\prime}))$ (with a binary search to find $k$). Thus, the overall computational complexity is $O(n^{\prime}\log(n^{\prime}))$. If we assume that each client has on average $K$ RATs, then on average $n^{\prime}$ would be equal to $\frac{KN}{M}$. Thus, the computational complexity would also be equal to $O(\frac{KN}{M}\log(\frac{KN}{M}))$.

Each BS uses the total throughput of each client across all its RATs in its calculations to find the water-fill level and the new $\lambda_{i,j}$s. Each time a client’s time resource (and hence total throughput) is changed, the client needs to inform all BSs to which it is connected about its new total throughput. Thus, the total message passing overhead generated by clients of a single BS is at most equal to $O(n^{\prime}K)$, or alternatively $O(\frac{K^{2}N}{M})$.

IV Convergence and Speed of AFRA

In this section, we investigate the convergence properties of AFRA. We first show that as BSs autonomously execute AFRA, the system converges to an equilibrium. Next, we investigate the convergence time properties of AFRA and provide tight bounds to quantify it.

IV-A Convergence to an Equilibrium

Before we discuss convergence, we present a formal definition of an equilibrium.

Definition 1**.**

*Equilibrium: The vector of time fractions across all the BSs and clients is an equilibrium outcome if none of the BSs can increase its water-fill level through unilateral change of its time resource allocations. *

Our next theorem guarantees the convergence of AFRA.

Theorem 1**.**

*Let each BS autonomously execute AFRA. Then, the system converges to an equilibrium, i.e., $\forall i\in\bf N$ and $j\in\bf M$ $\lambda_{i,j}$ $\to$ $\lambda_{i,j}^{eq}$, $\theta_{j}$ $\to$ $\theta_{j}^{eq}$, and $r_{i}$ $\to$ $r_{i}^{eq}$.

Proof.

Let $\bm{\lambda}$ denote the vector of time fractions ($\lambda_{i,j}$s) across all clients and BSs, and $f(\bm{\lambda})=\sum_{i=1}^{N}\omega_{i}\log(r_{i})$ be the potential function. A potential function [26] is a useful tool to analyze equilibrium properties, as it maps the payoff (e.g., throughput) of all clients into a single function.

Since the number of clients and BSs is finite, $f$ is bounded. The key step to prove convergence, is to show that each time a BS $j$ adjusts its time fractions (i.e., $\lambda_{i,j}$s), the potential function ($f$) increases. This property coupled with $f$’s boundedness guarantees its convergence. We will show later in Eq. (15) that the change in potential function is proportional to the product of the change in water-fill levels and the change in $\lambda_{i,j}$s. Since $f$ converges (i.e., its variations converge to 0), one or both of these terms should converge to 0. Either of these conditions guarantee the convergence of the $\lambda_{i,j}$s (and hence, $\theta_{j}$s and $r_{i}$s).

Next, we show that each time a BS runs AFRA, $f$ increases. When a BS runs AFRA, it takes some time resources from clients with high $\frac{r_{i}}{\omega_{i}R_{i,j}}$ and distributes them across clients with lower values. To ease the proof presentation, we focus on two clients and follow the changes on $f$ as the BS adjusts the $\lambda_{i,j}$s dedicated to these clients.

Let, $i,i^{\prime}$ denote two clients who are currently receiving time resources from BS $j$. Assume the following initial (old) order between these two clients

[TABLE]

Therefore, as BS $j$ executes AFRA it changes the time resources from $\lambda_{i}$ and $\lambda_{i^{\prime}}$ to $\lambda_{i}+\delta$ and $\lambda_{i^{\prime}}-\delta$, respectively. This, only changes the two corresponding terms in the potential function, i.e.

[TABLE]

Let $g(\delta)$ denote the variation in potential function, i.e.

[TABLE]

Thus, to prove convergence, we need to prove that $g(\delta)$ is always positive. We prove this by showing that first $g^{\prime}(\delta)\geq 0$. This shows that $g(\delta)$ is always non-decreasing. Second, we show that $g(\delta)$ is positive for very small values of $\delta$. Now

[TABLE]

Here $r_{i}^{new}$ and $r_{i^{\prime}}^{new}$ are the new throughput values for clients $i$ and $i^{\prime}$, respectively. It is clear that after BS $j$ adjusts the time resources, we still have $\frac{r^{new}_{i}}{\omega_{i}R_{i,j}}\leq\frac{r^{new}_{i^{\prime}}}{\omega_{i^{\prime}}R_{i^{\prime},j}}$. This is because after BS $j$ reduces $\lambda_{i^{\prime},j}$, $\frac{r_{i^{\prime}}^{new}}{\omega_{i^{\prime}}R_{i^{\prime},j}}$ would be either equal to the new water-fill level or higher than it (if $\lambda_{i^{\prime},j}=0$). On the other hand, $\frac{r_{i}^{new}}{\omega_{i}R_{i,j}}$ would be equal to the new water-fill level. As a result, the final term in Eq. (14) is non-negative. Finally, $g(\delta)$ is greater than zero for small values of $\delta$ because

[TABLE]

The last term in the above equation is due to Eq. (11).

IV-B Convergence Time

Before we can derive a bound on convergence time, we need to define a discretization factor on the time fractions (i.e., $\lambda_{i,j}$s). This technicality is due to the fact that $\lambda_{i,j}$s in our model are continuous variables, which can cause some BSs to continuously make infinitesimal adjustments to them. These adjustments converge to 0 as time goes to infinity.

In practice, operations always happen in discretized levels. For example, consider the following discretization policy:

Definition 2**.**

Discretization Policy: During water-fill calculation by a BS j in AFRA, the time fraction allocated to the client with minimum $\frac{r_{i}}{\omega_{i}R_{i,j}}$ should increase by at least $\epsilon$. Otherwise, the BS would not update its time fractions.

Based on the above discretization policy, we can derive the following bound on the convergence time.

Theorem 2**.**

*Consider a HetNet with N clients and M BSs. Then, the number of steps that it takes for AFRA to converge is upper bounded by O($\frac{NM^{2}\log(MN)}{\epsilon^{2}}$).

Proof.

Let $f(\bm{\lambda})=\sum_{i=1}^{N}\omega_{i}\log(r_{i})$ be the potential function from the proof of Theorem 1. To compute a bound on the convergence time, we study the increments of $f$. The key step is to find a lower bound on $f$’s increments. Since $f$ increases whenever a BS makes adjustments to its $\lambda_{i,j}$s, the convergence time is then upper bounded by the difference between the maximum and minimum possible values of $f$ divided by the lower bound on $f$’s increments.

We take the following steps to find a lower bound on the potential function’s increments. Let $\{i_{1},i_{2},...,i_{q}\}$ denote the set of clients with non-zero PHY rates to BS $j$ and assume the following initial (old) order among the clients

[TABLE]

When BS $j$ executes AFRA, it adjusts the time fractions in a way that increases the time resources allocated to client $i_{1}$. Let $\epsilon_{i_{1}}$ denote the increase in client $1$’s time resources and $r^{new}_{i_{1}}=r_{i_{1}}$ its new throughput. Let $\epsilon_{i_{p}}$ denote the change in client $i_{p}$’s ($i_{p}\in\{i_{2},...,i_{q}\}$) time resources and $r^{new}_{i_{p}}$ its new throughput. Hence, we have

[TABLE]

However, even after BS $j$ adjusts its time resources, $i_{1}$ would still have the minimum $\frac{r_{i}}{\omega_{i}R_{i,j}}$ across all clients. This is due to the water-fill based operation in AFRA. As a result

[TABLE]

Next, we find a lower bound on the potential function’s increments

[TABLE]

Let $W=\sum_{p=2}^{q}\omega_{i_{p}}$ and $x_{p}=\frac{\epsilon_{i_{p}}R_{i_{1},j}\omega_{i_{1}}}{r_{i_{1}}\omega_{i_{p}}}$. Since the logarithm is a concave function, from Jensen’s inequality [27],

[TABLE]

Leveraging Eq. (23), we conclude that Eq. (22) is

[TABLE]

where $z=\epsilon_{i_{1}}\frac{R_{i_{1},j}}{r_{i1}}$ and $\gamma=\frac{W}{\omega_{i_{1}}}$. Note that since we seek an upper bound on convergence time, we can choose a small enough $\epsilon_{i_{1}}$ so that $z,\frac{z}{\gamma}<1$. These assumptions increase the upper bound but allow us to use the Taylor series in Eq. (24). If we let $R_{min}$ and $R_{max}$ denote the minimum and maximum PHY rates across all the clients and BSs, then we have

[TABLE]

[TABLE]

[TABLE]

[TABLE]

V Optimality of AFRA

Beyond convergence, we study the optimality properties of AFRA’s equilibria. We first derive some useful properties of the equilibria that we leverage for optimality analysis. Next, we prove that the equilibria also maximize the global proportional fair resource allocation problem across all the BSs, and hence are globally optimal. Finally we discuss the uniqueness of the equilibria and prove that while the equilibrium throughput vector across all the clients is unique, there could be infinitely many resource allocations that realize this outcome. For simplicity, we do not consider discretization in this section.

Theorem 3**.**

*Consider an equilibrium outcome of AFRA. Let $r^{eq}_{i}$ denote the throughput of client i, $\theta^{eq}_{j}$ the water-fill level of BS j, and $\lambda^{eq}_{i,j}$ the fraction of time allocated to client i by BS j. Then

I $\frac{\omega_{i}R_{i,j}}{r^{eq}_{i}}\leq\frac{1}{\theta^{eq}_{j}}\hskip 7.22743pt\forall i\in\bf N,j\in\bf M$

II $\sum_{i=1}^{N}\lambda_{i,j}^{eq}=1\hskip 7.22743pt\forall j\in\bf M$

III $\sum_{i=1}^{N}\omega_{i}=\sum_{j=1}^{M}\frac{1}{\theta^{eq}_{j}}$

Proof.

Part 1. From the water-fill definition we have

[TABLE]

Property
I follows from Eqs. (26) and (27).

Part 2. Every BS can always increase its water-fill level by distributing its unused time resources across its clients. The property follows, since at equilibrium the water-fill levels cannot be further increased.

Part 3. We leverage
I and
II to derive property
III as follows

[TABLE]

We next show that any equilibrium outcome of AFRA is globally optimal, i.e., it maximizes the global PF resource allocation problem.

Theorem 4**.**

*Consider an equilibrium outcome of AFRA. Then, the equilibrium outcome also maximizes the global PF resource allocation problem, i.e., it maximizes $\sum_{i=1}^{N}\omega_{i}log$($r_{i}$) subject to the feasibility constraints in Eqs. (1)-(3).

Proof.

Let $r^{eq}_{i}$ and $\theta^{eq}_{j}$ denote the throughput of client $i$ and water-fill level of BS $j$ at an equilibrium, respectively.

We prove that for any feasible selection of $\lambda_{i,j}$s (i.e., $\lambda_{i,j}$s that satisfy the feasibility conditions in Eqs. (2) and (3)) and the corresponding clients’ throughput values (i.e., $r_{i}$s as defined in Eq. (1)) we have

[TABLE]

Define $\textrm{W}=\sum_{i=1}^{N}\omega_{i}$. Eq. (29) can then be proved through the following inequalities by leveraging properties
I and
III from Theorem 3:

[TABLE]

In our last theorem we prove that while the equilibrium throughput vector across all clients is unique, there could be infinitely many resource allocations that realize this outcome.

Theorem 5**.**

*Let ${\bf{r}}^{eq}$ = $(r_{1}^{eq},...,r_{N}^{eq})$ denote the vector of throughput rates across all clients at an equilibrium. Then, ${\bf{r}}^{eq}$ is unique. However, there could be infinitely many resource allocations across the BSs that realize ${\bf{r}}^{eq}$.

Proof.

Part 1. We first prove that ${\bf{r}}^{eq}$ is unique. Let ${\bf{r}}^{eq}$ maximize the global proportional-fair resource allocation across all clients and assume ${\bf{r}}^{\prime eq}$ is a different equilibria. From Theorem 4, we know that every other equilibrium should also maximize the global PF resource allocation. This means that all inequalities in Eq. (30) should be equalities for any equilibrium, including ${\bf{r}}^{\prime eq}$. Now, for the first inequality to be an equality (i.e., Jensen inequality of Eq. (30)), the following condition needs to be satisfied [27]

[TABLE]

Further, since $\sum_{i=1}^{N}\omega_{i}\log(r^{eq}_{i})=\sum_{i=1}^{N}\omega_{i}\log(r^{\prime eq}_{i})$, we conclude that

[TABLE]

Part 2. To prove that there could be infinitely many resource allocations that realize ${\bf{r}}^{eq}$, we provide an example. Consider a topology with two BSs ($j_{1}$, $j_{2}$) and two clients ($i_{1},i_{2}$). Let $R_{i_{1},j}=1\hskip 7.22743pt\forall j\in\bf M$, $R_{i_{2},j}=2\hskip 7.22743pt\forall j\in\bf M$, and $\omega_{i_{1}}=\omega_{i_{2}}=2$. Then, $\sum\omega_{i}\log(r_{i})$ is maximized by the following time fractions for any $\alpha\in[0\hskip 7.22743pt1]$.

[TABLE]

Here, irrespective of $\alpha$, $r_{i_{1}}=1$ and $r_{i_{2}}=2$.

VI Performance Evaluation

In this section, we evaluate AFRA’s performance through experiments and simulations. First, we investigate the benefits of MAC level traffic aggregation in a small testbed composed of four SDR (software-defined radio)-based BSs and clients. Next, we conduct simulations to evaluate AFRA’s equilibria properties as we scale the number of clients and BSs. Finally, we compare AFRA’s speed and over-the-air signaling overhead against DDNUM, a dual decomposition based algorithm that we derived from the NUM framework.

VI-A SDR-Based Implementation and Real-World Performance

Implementation. We construct a HetNet topology composed of a WiFi BS, a cellular BS, and two clients. The two BSs are physically separated from each other and are placed in an indoor lab environment (Fig. 4(a)). We use a WARP board [28] with 802.11a reference design as our WiFi BS. We use another WARP board with OFDM PHY (WARP OFDM reference design) and a custom TDMA (Time Division Multiple Access) MAC to mimic a cellular BS. We use two other WARP boards to construct our two clients. Each client has access to both WiFi and cellular radios, and remains static and connected to both BSs throughout the experiments.

A server running iPerf sessions is connected to both BSs through Ethernet. For each client, the server generates a single fully-backlogged UDP traffic flow with 500 byte packets. We implement a below-IP sublayer to split this traffic flow between the two BSs. This sublayer is responsible for selection of the BS to be used for each packet, and acts similar to the LWA Adaptation Protocol (LWAAP) in the LWA standard [2]. In our implementation, we sequentially iterate between the WiFi and cellular BSs to route the packets of each traffic flow.

AFRA, as presented in Section III-D, does not account for various types of overhead (e.g., PHY/MAC header, ACKs, idle slots, collisions) that exist in PHY/MAC protocols. To address the issue, we introduce the notion of effective rate ($R^{\text{eff}}$) and replace all $R_{i,j}$s in AFRA with $R^{\text{eff}}_{i,j}$s. For a single packet, $R^{\text{eff}}$ can be calculated as the number of bits in the packet divided by the total time it takes by a BS to successfully transmit that packet (including all overhead). In our implementation, each BS keeps track of the total time spent in successfully transmitting the past 5 packets of each traffic flow (i.e., the past 5 packets of each client) to calculate its $R^{\text{eff}}_{i,j}$. The averaging over 5 packets is to account for channel fluctuations in our experiments, and can be adjusted based on the client mobility.

We implement the following mechanisms: (i) WiFi only: the cellular BS is off but the WiFi BS is active; (ii) Cellular only: WiFi BS is off; (iii) AGG-RR: this scheme uses aggregation but with a round robin (RR) scheduler at the WiFi BS and conventional PF MAC at the cellular BS. With the RR scheduler, the WiFi BS maintains a different queue for each client and sequentially serves a single packet from each queue at every round. With the PF MAC at the cellular BS, the BS dedicates its time resources to each client according to Section III-C (single BS PF); (iv) AFRA: each BS uses its calculated $\lambda_{i,j}$s to determine the number of packets that should be served from each queue in WiFi and the number of time slots that should be dedicated to each queue (client) in cellular, at every round. In our implementation, both clients’ $\omega_{i}$ are equal to 1 and the BSs updates their $\lambda_{i,j}$s every 5 ms.

Performance Results. Fig. 4(c) shows the performance of the four schemes. In both the WiFi only and Cellular only options, only a single BS is active throughout the experiments. We observe that the Cellular only scheme provides a higher sum throughput than the WiFi only scheme. With careful evaluation of packet transmission traces, we discovered that this higher throughput is primarily due to the corresponding MAC protocols. In particular, WiFi MAC provides the same transmission opportunity to each traffic flow (client). As a result, the client with lower PHY rate occupies the channel for a longer duration that the other client. This decreases the throughput for both clients. In contrast, the cellular TDMA MAC provides the same transmission time for both clients (with 2 clients, single BS PF equally divides the time between the clients (Eq. 5)). As a result, the throughput of the client with higher PHY rate does not drop because of the client with a lower PHY rate. This, along with other MAC issues such as WiFi contention reduce the WiFi only throughput.

Fig. 4(c) also shows that the two RAT aggregation schemes (AGG-RR and AFRA) can successfully aggregate WiFi and cellular capacities and provide a higher sum throughput than the WiFi only and Cellular only options. Further, AFRA increases the average total throughput by 45% (from 20 to 29 Mbps) with 18 and 11 Mbps per-client total throughput values (per-client throughput plots are shown in Fig. 4(d)). Let us define the proportional fairness index as PF = $\sum_{i=1}^{2}\log(r_{i})$ ($r_{i}$ is the total throughput of each client across its RATs in Mbps). Then, the PF index in AFRA would be 2.3. With AGG-RR, the per-client throughput rates drop to 12.5 and 7.5 Mbps. Thus, the PF index reduces to 1.97. AGG-RR uses the conventional scheduling algorithms on each BS (i.e., it uses RR in WiFi and single BS PF in cellular), which reduce both the sum throughput and the PF fairness index.

VI-B AFRA’s Equilibria Properties

Setup. We simulated network deployments with N clients and M BSs to evaluate AFRA’s equilibria properties as we scale the number of clients and BSs. All clients’ $\omega_{i}$s are equal to 1. Half of the BSs are WiFi and the other half are cellular. Each client has access to 4 RATs, two WiFi and two cellular. The PHY rates for the WiFi and cellular RATs are randomly selected from the sets $\{1,2,5.5,11\}$ Mbps and $\{5.2,10.3,25.5,51\}$ Mbps, respectively. In each simulation realization, we randomly associate clients’ RATs with BSs. Next, we run AFRA until an equilibrium is reached. We set the discretization factor $\epsilon$ equal to 0.05, i.e., a BS adjusts its time fractions only if the increase in time fraction (i.e., $\lambda_{i,j}$) at its client with minimum $\frac{r_{i}}{\omega_{i}R_{i,j}}$ is greater than or equal to 0.05. For the initial allocation, each BS equally divides its time across its clients. Unless otherwise specified, each of our simulation points is an average of 100 simulation realizations.

AFRA’s Convergence Time. Figs. 5(a) and 5(b) depict the impact of the number of clients and BSs on AFRA’s convergence time. In each of these figures, we count the number of steps until convergence is reached. At each step, a single BS that needs to adjust its time fractions is randomly selected. In Fig. 5(a), we vary the number of clients from 10 to 100 and plot the corresponding convergence times for three different M values: 10, 20, and 50. We repeat this simulation by changing the N and M variables and plot the corresponding results in Fig. 5(b). From these two figures, we observe that time to convergence is highest when the number of clients is between one to two times the number of BSs. As the ratio between the number of clients and BSs (i.e., $\frac{N}{M}$) leaves this range, the convergence time rapidly drops and then stabilizes. The results show that AFRA requires a small number of steps to reach an equilibrium.

Policies to Further Reduce AFRA’s Convergence Time. Our next goal is to design policies that can further reduce AFRA’s convergence time. To gain intuition on how to design such policies, we simulated a topology with 10 clients and 10 BSs and plotted the evolution of the potential function (i.e., $\sum_{i}\log(r_{i})$) as BSs adjusted their time fractions. The results are shown in Fig. 5(c). Here, each Run corresponds to a different simulation realization. From these realizations we make two observations. First, there is a wide gap in the convergence times. Second, a high jump in the potential function pushes the system closer to equilibrium. Based on these observations, we designed a prioritization policy among the BSs to reduce the convergence time.

We let each BS calculate the increase in the potential function assuming that it is the only BS executing AFRA. Since in AFRA each BS knows the current total throughput of its clients, it has all the needed information to calculate the increase in the potential function due to its action. Next, each BS broadcasts its calculated value. Finally, the BS with the highest value gets priority in executing AFRA. This distributed policy can be easily implemented in networks where all the BSs are connected to the same backbone (e.g., Ethernet). The solid black curve in Fig. 5(c) shows the potential function’s evolution with this policy. We observe that on average, the convergence time drops from 15 steps to 10, i.e., the prioritization policy reduces the convergence time by 33%. We repeated this simulation for another setup with 20 clients to increase the topological redundancy. The results are plotted in Fig. 5(d). Similarly, the average convergence time reduces from 19 steps to 13, i.e., a 32% reduction in convergence time.

VI-C Comparison Against DDNUM

We have compared AFRA’s performance against DDNUM, a distributed algorithm that we developed by leveraging dual decomposition and the NUM framework. Dual decomposition is appropriate to solve the multi-RAT PF allocation problem, because the coupling constraint (Eq. (2)) can be relaxed through the dual problem and then the problem decouples into subproblems that can be iteratively solved by clients and BSs.

DDNUM is in essence similar to the standard dual algorithm presented in [7] to solve the basic NUM problem. We modified the algorithm in [7] to capture the constraints of our problem. At a high level, DDNUM has three main steps (for detailed algorithm derivation and discussions, refer to the Appendix):

• •

Step 1: Initialization: set $t=0$ and $\boldsymbol{\mu}(0)$ to some nonnegative value for each BS. Here, $\boldsymbol{\mu}(t)$ is the vector of Lagrange multipliers that shows the cost or congestion across all BSs. Each BS broadcasts its $\mu_{j}(0)$ to clients with $R_{i,j}>0$.

• •

Step 2: Each client $i$ locally solves its Lagrangian problem, i.e., finds its time fractions ($\lambda^{*}_{i,j}(\mu_{j}(t))$) for each BS with $R_{i,j}>0$ and informs those BSs.

• •

Step 3: Each BS updates its price with a step size $\gamma$ and broadcasts the new price $\mu_{j}(t+1)$ to all its clients.

This procedure is repeated until a satisfying termination point is reached (e.g., the solution is within a desired proximity of the optimal solution). Similar to AFRA, DDNUM is guaranteed to converge and maximize the global optimization problem. However, there are several practicality and performance issues. We highlight a few of these issues next.

Setup. To compare AFRA to DDNUM, we used the simulation setup in Section VI-B (without the BS prioritization policy). We first run AFRA and let the system converge to an equilibrium. Next, we consider the 95% value of AFRA’s potential function at equilibrium as the desired algorithm termination point. We count the number of steps to reach the termination point and the resulting over-the-air signaling overhead in each of these two schemes. In DDNUM, the step size $\gamma$ (step 3) provides a balance between the final throughput values and speed. We choose the $\gamma$ that results in the fastest convergence time, subject to the potential function reaching the termination point. Finally, both AFRA and DDNUM can operate in either parallel or sequential mode with similar relative performance. We present the sequential mode results, i.e., at each time only a single BS adjusts its water-fill level (in AFRA) or announces a new price (in DDNUM). We assume that clients immediately update their BSs about their new throughput values (in AFRA) and desired $\lambda_{i,j}$s (in DDNUM) with no impact on the convergence time (similar to an FDD system in which uplink data is immediately available).

Speed. Fig. 6(a) show the convergence time results for a scenario with 10 BSs and varying number of clients. We observe that irrespective of the number of clients, DDNUM increase the convergence time by a factor of 2-3x with an average of 2.4x. In AFRA, each BS simultaneously calculates the water-fill level and finds the corresponding time fraction for each client. In DDNUM, the pricing mechanism requires a high number of iterations so that clients can find their optimal time fractions. This increases the convergence time.

Over-the-Air Overhead. Fig. 6(b) shows the wireless signaling overhead results of the two schemes. We observe that DDNUM increases the signaling overhead by a factor of 4-5x with an average of 4.5x. There are several factors that contribute to DDNUM’s high signaling overhead. First, the increases in convergence time results in a similar multiplicative increase in overhead. Second, in DDNUM both BSs and clients contribute to overhead. BSs continuously broadcast new prices and clients continuously inform each of their BSs about their desired time fractions. In contrast, in AFRA only clients update the BSs regarding their new throughput values. Third, with careful examination of simulation traces, we observed that in AFRA the water-fill operation only impacts a few of a BS’s clients each time. In DDNUM, each time a BS updates its price, most of its clients would request new time fractions.

Practicality. In DDNUM, each BS broadcasts its price while each client finds its desired $\lambda^{*}_{i,j}$s from its BSs. However, in real wireless systems BSs are responsible for resource allocation. Note that in DDNUM, it is not practical to shift the calculation of $\lambda^{*}_{i,j}$s (i.e., step 2) to BSs. This is because in order for a BS $j$ to find the $\lambda^{*}_{i,j}$s for each of its clients (e.g., $i$), it would require knowledge about the client’s $R_{i,j}$ and $\mu_{j}$ to every other BS for which the client’s rate (i.e., $R_{i,j}$) is greater than zero. This information is only available at the client and pushing it to the BS would significantly increase the overhead, which is already very high in DDNUM.

Complexity. In DDNUM, each client has to solve a complex Lagrangian subproblem to find its desired time fraction for each BS (step 2). This increases the computational complexity on the client devices. In contrast, AFRA identifies the time resources at the BSs, which have higher power and computing resources. Moreover, as we discussed in Section III-D, AFRA has a very low total computational complexity.

VII Conclusion

We addressed the problem of proportional fair multi-RAT traffic aggregation in HetNets. We studied the conventional PF resource allocation in a single BS and showed that we can look at the problem as a special type of water-filling. Based on this observation, we designed a new fully distributed water-filling algorithm for HetNets. We also studied the convergence, speed, and optimality of our algorithm. We proved that our algorithm quickly converges to equilibria and derived tight bounds to quantify its speed. We also studied the characteristics of the optimal outcome, and used the properties to prove the outcomes of our algorithm are globally optimal.