Energy Harvesting Networks with General Utility Functions: Near Optimal Online Policies

TL;DR

This paper develops near-optimal online scheduling policies for energy harvesting communication systems with general utility functions, demonstrating their effectiveness across various energy arrival distributions and system parameters.

Contribution

It characterizes the optimal policies for Bernoulli arrivals and shows fixed fraction policies are near-optimal for general i.i.d. energy arrivals, extending previous results.

Findings

Fixed fraction policies are within a constant multiplicative gap of optimal for all energy arrivals.

Under certain utility function conditions, fixed fraction policies are within a constant additive gap of optimal.

The analysis applies to systems with finite battery sizes and i.i.d. energy arrivals.

Abstract

We consider online scheduling policies for single-user energy harvesting communication systems, where the goal is to characterize online policies that maximize the long term average utility, for some general concave and monotonically increasing utility function. In our setting, the transmitter relies on energy harvested from nature to send its messages to the receiver, and is equipped with a finite-sized battery to store its energy. Energy packets are independent and identically distributed (i.i.d.) over time slots, and are revealed causally to the transmitter. Only the average arrival rate is known a priori. We first characterize the optimal solution for the case of Bernoulli arrivals. Then, for general i.i.d. arrivals, we first show that fixed fraction policies [Shaviv-Ozgur] are within a constant multiplicative gap from the optimal solution for all energy arrivals and battery sizes. We then derive a set of sufficient conditions on the utility function to guarantee that fixed fraction policies are within a constant additive gap as well from the optimal solution.

Equations

$$0\le g_t\le b_t\triangleq \min \{b_{t-1}-g_{t-1}+E_t,B\},\forall t$$

$${\mathcal{U}}_n(\mathbf{g})\triangleq \frac{1}{n}\mathbb{E}\left[\sum _{t=1}^{n}u\left(g_t\right)\right]$$

$${\rho }^{\ast }\triangleq \underset{\mathbf{g}\in \mathcal{F}}{\max }\underset{n\to \infty }{\lim }{\mathcal{U}}_n(\mathbf{g})$$

$$q\triangleq \frac{\mathbb{E}[E_t]}{B}$$

$${\tilde{g}}_t=q{b}_t$$

$${\rho }^{\ast }\le u(\mu )$$

$$\frac{1}{2}\le \frac{\rho \left(\tilde{\mathbf{g}}\right)}{u\left(\mu \right)}\le 1$$

$$h_{\theta }(x)\triangleq u(\theta x)-u(x)$$

$$h(\theta )\triangleq \underset{x}{\inf }{h}_{\theta }(x)$$

$$r\triangleq (1-q)\underset{t\to \infty }{\lim }\frac{1-{\lim }_{x\to {\bar{x}}_{t+1}}u\left((1-q{)}^{t+1}x\right)/u(x)}{1-{\lim }_{x\to {\bar{x}}_t}u\left((1-q{)}^{t}x\right)/u(x)}<1$$

$$u\left(\mu \right)+\alpha \le \rho \left(\tilde{\mathbf{g}}\right)\le u\left(\mu \right)$$

$$\underset{\mu \to \infty }{\lim }\rho \left(\tilde{\mathbf{g}}\right)={\rho }^{\ast }$$

$${\hat{E}}_t\in \{0,B\},\text{with}\mathbb{P}[{\hat{E}}_t=B]=p,\text{and}pB=\mu$$

$$\underset{n\to \infty }{\lim }{\hat{\mathcal{U}}}_n(\mathbf{g})$$

$$=\frac{1}{\mathbb{E}[L]}\mathbb{E}\left[\sum _{t=1}^{L}u\left(g_t\right)\right]\text{a.s.}$$

$${\tilde{g}}_t$$

$$\underset{\mathbf{g}}{\max }$$

$$\sum _{t=1}^{\infty }{g}_t\le B,g_t\ge 0,\forall t$$

$$\mathcal{L}=-\sum _{t=1}^{\infty }p(1-p{)}^{t-1}u\left(g_t\right)+\lambda \left(\sum _{t=1}^{\infty }{g}_t-B\right)-\sum _{t=1}^{\infty }{\eta }_t{g}_t$$

$$u^{\prime }\left(g_t\right)=\frac{\lambda -{\eta }_t}{p(1-p{)}^{t-1}}$$

$$g_t=f\left(\frac{\lambda }{p(1-p{)}^{t-1}}\right)$$

$$g_t=f\left(\frac{\lambda }{p(1-p{)}^{t-1}}\right)>0\iff \lambda <p(1-p{)}^{t-1}{u}^{\prime }(0)$$

$$\sum _{t=1}^{\infty }f\left(\frac{\lambda }{p(1-p{)}^{t-1}}\right)=B$$

$$\lambda \ge p(1-p{)}^N{u}^{\prime }(0)$$

$$\sum _{t=1}^{N}f\left(\frac{\lambda }{p(1-p{)}^{t-1}}\right)=B$$

$$\sum _{t=1}^n{g}_t\le \sum _{t=1}^n{E}_t,\forall n$$

$$\frac{1}{n}\sum _{t=1}^n{g}_t\le \mu +\delta \text{a.s.}$$

$$\frac{1}{n}\sum _{t=1}^{n}u(g_t)$$

$$\underset{n\to \infty }{\lim }{\hat{\mathcal{U}}}_n(\tilde{\mathbf{g}})$$

$$=\sum _{t=1}^{\infty }p(1-p{)}^{t-1}u\left((1-p{)}^{t-1}\mu \right)$$

$$\stackrel{(b)}{\ge }\sum _{t=1}^{\infty }p(1-p{)}^{2(t-1)}u\left(\mu \right)$$

$$=\frac{1}{2-p}u(\mu )\ge \frac{1}{2}u(\mu )$$

$$\rho (\tilde{\mathbf{g}})\ge \underset{n\to \infty }{\lim }{\hat{\mathcal{U}}}_n(\tilde{\mathbf{g}})$$

Full text

I Introduction

A single-user communication channel is considered, where the transmitter relies on energy harvested from nature to send its messages to the receiver. The transmitter has a battery of finite size to save its incoming energy, and achieves a reward for every transmitted message that is in the form of some general concave increasing utility function of the transmission power. The goal is to characterize online power control policies that maximize the long term average utility subject to energy causality constraints.

Offline power scheduling in energy harvesting communication systems has been extensively studied in the recent literature. Earlier works [2, 3, 4, 5] consider the single-user setting. References [6, 7, 8, 9, 10, 11, 12] extend this to broadcast, multiple access, and interference settings; and [13, 14, 15, 16] consider two-hop and relay channels. Energy cooperation and energy sharing concepts are studied in [17, 18]. References [19, 20, 21, 22, 23, 24] study energy harvesting receivers, where energy harvested at the receiver is spent mainly for sampling and decoding. Other works [24, 25, 26, 27, 28, 29] study the impact of processing costs, i.e., the power spent for circuitry, on energy harvesting communications.

Recently, [1] has introduced an online power control policy for a single-user energy harvesting channel that maximizes the long term average throughput under the AWGN capacity utility function $\frac{1}{2}\log(1+x)$. The proposed policy is near optimal in the sense that it performs within constant multiplicative and additive gaps from the optimal solution that is independent of energy arrivals and battery sizes. This is extended to broadcast channels in [30], multiple access channels in [31, 32], and systems with processing costs in [33, 34] (for examples of earlier online approaches see, e.g., [35, 36, 37]).

In this paper, we generalize the approaches in [1] to work for general concave monotonically increasing utility functions for single-user channels. That is, we consider the design of online power control policies that maximize the long term average general utilities. One motivation for this setting is energy harvesting receivers. Since power consumed in decoding is modelled as a convex increasing function of the incoming rate [19, 20, 23], the rate achieved at the receiver is then a concave increasing function of the decoding power. In our setting, energy is harvested in packets that follow an i.i.d. distribution with amounts known causally at the transmitter. The transmitter has a finite battery to store its harvested energy. We first study the special case of Bernoulli energy arrivals that fully recharge the battery when harvested, and characterize the optimal online solution. Then, for the general i.i.d. arrivals, we show that the policy introduced in [1] performs within a constant multiplicative gap from the optimal solution for any general concave increasing utility function, for all energy arrivals and battery sizes. We then provide sufficient conditions on the utility function to guarantee that such policy is within a constant additive gap from the optimal solution.

II System Model and Problem Formulation

We consider a single-user channel where the transmitter relies on energy harvested from nature to send its messages to the receiver. Energy arrives (is harvested) in packets of amount $E_{t}$ at the beginning of time slot $t$. Energy packets follow an i.i.d. distribution with a given mean. Our setting is online: the amounts of energy are known causally in time, i.e., after being harvested. Only the mean of the energy arrivals is known a priori. Energy is saved in a battery of finite size $B$.

Let $u$ be a differentiable, concave, and monotonically increasing function representing some general utility (reward) function, with $u(0)=0$ and $u(x)>0$ for $x>0$, and let $g_{t}$ denote the transmission power used in time slot $t$. By allocating power $g_{t}$ in time slot $t$, the transmitter achieves $u\left(g_{t}\right)$ instantaneous reward. Denoting $\mathcal{E}^{t}\triangleq\{E_{1},E_{2},\dots,E_{t}\}$, a feasible online policy ${\bm{g}}$ is a sequence of mappings $\{g_{t}:~{}\mathcal{E}^{t}\rightarrow\mathbb{R}_{+}\}$ satisfying

[TABLE]

with $b_{1}\triangleq B$ without loss of generality (using similar arguments as in [1, Appendix B]). We denote the above feasible set by $\mathcal{F}$. Given a feasible policy ${\bm{g}}$, we define the $n$-horizon average reward as

[TABLE]

Our goal is to design online power scheduling policies that maximize the long term average reward subject to (online) energy causality constraints. That is, to characterize

[TABLE]

III Main Results

In this section, we present the main results of this paper. We note that problem (3) can be solved by dynamic programming techniques since the underlying system evolves as a Markov decision process. However, the optimal solution using dynamic programming is usually computationally demanding with few structural insights. Therefore, in the sequel, we aim at finding relatively simple online power control policies that are provably within a constant additive and multiplicative gap from the optimal solution for all energy arrivals and battery sizes.

We assume that $E_{t}\leq B~{}\forall t$ a.s., since any excess energy above the battery capacity cannot be saved or used. Let $\mu=\mathbb{E}[E_{t}]$, where $\mathbb{E}[\cdot]$ is the expectation operator, and define

[TABLE]

Then, we have $0\leq q\leq 1$ since $E_{t}\leq B$ a.s. We define the power control policy as follows [1]

[TABLE]

That is, in each time slot, the transmitter uses a fixed fraction of its available energy in the battery. Such policies were first introduced in [1], and coined fixed fraction policies (FFP). Clearly such policies are always feasible since $q\leq 1$. Let $\rho\left(\tilde{{\bm{g}}}\right)$ be the long term average utility under the FFP $\{\tilde{g}_{t}\}$. We now state the main results.

Lemma 1

The optimal solution of problem (3) satisfies

[TABLE]

Theorem 1

The achieved long term average utility under the FFP in (5) satisfies

[TABLE]

We note that the results in Lemma 1 and Theorem 1 indicate that the FFP in (5) achieves a long term average utility that is within a constant multiplicative gap from the optimal solution that is equal to $\frac{1}{2}$. This result is proved in [1] for $u(x)=\frac{1}{2}\log(1+x)$. Here, we are generalizing it to work for any concave increasing function $u$ with $u(0)=0$.

Next, we state the additive gap results. We first define

[TABLE]

for some $0\leq\theta\leq 1$, and define the following two classes of utility functions.

Definition 1** **(Utility Classes)

A utility function $u$ belongs to class $(A)$ if $h_{\theta}(x)$ does not converge to 0 as $x\rightarrow\infty$, and belongs to class $(B)$ if $\lim_{x\rightarrow\infty}h_{\theta}(x)=0$.

Now let us define the following function for $0<\theta<1$

[TABLE]

whenever the infimum exists. Note that the infimum exists for class $(B)$ utility functions since $h_{\theta}(x)<0$ for $x>0$ by monotonicity of $u$, and $h_{\theta}(0)=0$. We state some properties of the function $h$ in the next lemma. The proof follows by monotonicity and concavity of $u$ and is omitted for brevity.

Lemma 2

$h(\theta)$* is non-positive, concave, and non-decreasing in $\theta$.*

The next two theorems summarize the additive gap results for utility functions in classes $(A)$ and $(B)$ in Definition 1.

Theorem 2

If $h(\theta)$ exists, and if

[TABLE]

where $\bar{x}_{t}\in\arg\inf_{x}h_{(1-q)^{t}}(x)$; then the achieved long term average utility under the FFP in (5) satisfies

[TABLE]

where $\alpha\triangleq\sum_{t=0}^{\infty}q(1-q)^{t}h\left((1-q)^{t}\right)$ is finite.

Theorem 3

For class $(B)$ utility functions, the achieved long term average utility under the FFP in (5) satisfies

[TABLE]

We note that the results in Lemma 1 and Theorem 2 indicate that the FFP in (5) achieves a long term average utility, under some sufficient conditions, that is within a constant additive gap from the optimal solution that is equal to $\left|\sum_{t=0}^{\infty}q(1-q)^{t}h\left((1-q)^{t}\right)\right|$. One can further make this gap independent of $q$ by minimizing it over $0\leq q\leq 1$. We discuss examples of the above results in Section VI, where we also comment on FFP performance under utility functions that do not satisfy the sufficient conditions in Theorem 2.

IV Bernoulli Energy Arrivals

In this section, we characterize the optimal solution of a special case of the energy arrival i.i.d. process: the Bernoulli process. Let $\{\hat{E}_{t}\}$ be a Bernoulli energy arrival process with mean $\mu$ as follows

[TABLE]

where $\mathbb{P}[A]$ denotes the probability of $A$. Note that under such specific energy arrival setting, whenever an energy packet arrives, it completely fills the battery, and resets the system. This constitutes a renewal. Then, by [38, Theorem 3.6.1] (see also [1]), the following holds for any power control policy ${\bm{g}}$

[TABLE]

where $\hat{\mathcal{U}}_{n}({\bm{g}})$ is the $n$-horizon average utility under Bernoulli arrivals, and $L$ is a random variable denoting the inter-arrival time between energy arrivals, which is geometric with parameter $p$, and $\mathbb{E}[L]=1/p$.

Using the FFP defined in (5) in (IV) gives a lower bound on the long term average utility. Note that by (13), the fraction $q$ in (4) is now equal to $p$. Also, the battery state decays exponentially in between energy arrivals, and the FFP is

[TABLE]

for all time slots $t$, where the second equality follows since $pB=\mu$. Using (IV), problem (3) in this case reduces to

[TABLE]

which is a convex optimization problem. The Lagrangian is,

[TABLE]

where $\lambda$ and $\{\eta_{t}\}$ are Lagrange multipliers. Taking derivative with respect to $g_{t}$ and equating to 0 we get

[TABLE]

Since $u$ is concave, then $u^{\prime}$ is monotonically decreasing and $f\triangleq(u^{\prime})^{-1}$ exists, and is also monotonically decreasing. By complementary slackness, we have $\eta_{t}=0$ for $g_{t}>0$, and the optimal power in this case is given by

[TABLE]

and it now remains to find the optimal $\lambda$. We note by monotonicity of $f$, $\{g_{t}\}$ is non-increasing, and it holds that

[TABLE]

Hence, if $u^{\prime}(0)$ is infinite, then (20) is satisfied $\forall t$, and the optimal power allocation sequence is an infinite sequence. In this case, we solve the following equation for the optimal $\lambda$

[TABLE]

which has a unique solution by monotonicity of $f$.

On the other hand, for finite $u^{\prime}(0)$, there exists a time slot $N$, after which the second inequality in (20) is violated since $\lambda$ is a constant and $p(1-p)^{t-1}$ is decreasing. In this case the optimal power allocation sequence is only positive for a finite number of time slots $1\leq t\leq N$. We note that $N$ is the smallest integer such that

[TABLE]

Thus, to find the optimal $N$ (and $\lambda$), we first assume $N$ is equal to some integer $\{2,3,4,\dots\}$, and solve the following equation for $\lambda$

[TABLE]

We then check if (22) is satisfied for that choice of $N$ and $\lambda$. If it is, we stop. If not, we increase the value of $N$ and repeat. This way, we reach a KKT point, which is sufficient for optimality by convexity of the problem [39]. We note that for $u(x)=\frac{1}{2}\log(1+x)$ whose $u^{\prime}(0)$ is finite, [1] called $N$, $\tilde{N}$. We generalize their analysis for any concave increasing function $u$. This concludes the discussion of the optimal solution in the case of Bernoulli energy arrivals.

V General i.i.d. Energy Arrivals:

Proofs of Main Results

V-A Proof of Lemma 1

In this section, we derive the upper bound in Lemma 1 that works for all i.i.d. energy arrivals. Following [1] and [33], we first remove the battery capacity constraint setting $B=\infty$. This way, the feasible set $\mathcal{F}$ becomes

[TABLE]

Then, we remove the expectation and consider the offline setting of problem (3), i.e., when energy arrivals are known a priori. Since the energy arrivals are i.i.d., the strong law of large numbers indicates that $\lim_{n\rightarrow\infty}\frac{1}{n}\sum_{t=1}^{n}E_{t}=\mu$ a.s., i.e., for every $\delta>0$, there exists $n$ large enough such that $\frac{1}{n}\sum_{t=1}^{n}E_{t}\leq\mu+\delta$ a.s., which implies by (24) that the feasible set, for such $(\delta,n)$ pair, is given by

[TABLE]

Now fix such $(\delta,n)$ pair. The objective function is given by

[TABLE]

Since $u$ is concave, the optimal power allocation minimizing the objective function is $g_{t}=\mu+\delta$, $1\leq t\leq n$ [39] (see also [2]). Whence, the optimal offline solution is given by $u(\mu+\delta)$. We then have $\rho^{*}\leq u(\mu+\delta)$. Since this is true $\forall\delta>0$, we can take $\delta$ down to 0 by taking $n$ infinitely large.

V-B Proof of Theorem 1

We first derive a lower bound on the long term average utility for Bernoulli energy arrivals under the FFP as follows

[TABLE]

where $(a)$ follows by (IV), $(b)$ follows by concavity of $u$ [39], and the last inequality follows since $0\leq p\leq 1$. Next, we use the above result for Bernoulli arrivals to bound the long term average utility for general i.i.d. arrivals under the FFP in the following lemma; the proof follows by concavity and monotonicity of $u$, along the same lines of [1, Section VII-C], and is omitted for brevity.

Lemma 3

Let $\{\hat{E}_{t}\}$ be a Bernoulli energy arrival process as in (13) with parameter $q$ as in (4) and mean $qB=\mu$. Then, the long term average utility under the FFP for any general i.i.d. energy arrivals, $\rho(\tilde{\bm{g}})$, satisfies

[TABLE]

Using Lemma 1, (28), and Lemma 3, we have

[TABLE]

V-C Proof of Theorem 2

By Lemma 1 and Lemma 3, it is sufficient to study the lower bound in the case of Bernoulli arrivals. By (27) we have

[TABLE]

where $(c)$ follows since $h(\theta)$ exists, and is by definition no larger than $h_{\theta}(x)$, $\forall x,\theta$. Now to check whether $\alpha$ is finite, we apply the ratio test to check the convergence of the series $\sum_{t=0}^{\infty}(1-p)^{t}h\left((1-p)^{t}\right)$. That is, we compute

[TABLE]

where the second equality follows by definition of $h$. Next, we replace $\inf_{x}$ by $\lim_{x\rightarrow\bar{x}_{t}}$ since $\bar{x}_{t}\in\arg\inf h_{(1-p)^{t}}(x)$, and take the limit inside (after the 1). Finally, if $r<1$ then $\alpha$ is finite; if $r>1$ then $\alpha=-\infty$; and if $r=1$ then the test is inconclusive and one has to compute $\lim_{T\rightarrow\infty}\sum_{t=0}^{T}p(1-p)^{t}h\left((1-p)^{t}\right)$ to get the value of $\alpha$.

V-D Proof of Theorem 3

For utility functions of class $(B)$, we have $\lim_{x\rightarrow\infty}u(\theta x)-u(x)=0$. Thus, $\forall\epsilon>0$ there exists $\bar{\mu}$ large enough such that

[TABLE]

whence, for Bernoulli energy arrivals we have

[TABLE]

It then follows by Lemma 1 and Lemma 3 that

[TABLE]

and we can take $\epsilon$ down to 0 by taking $\mu$ infinitely large.

VI Examples and Discussion

In this section we present some examples to illustrate the results of this work. We first show that the utility function $u(x)=\frac{1}{2}\log(1+x)$ considered in [1] belongs to class $(A)$. Indeed we have $h_{\theta}^{\prime}(x)=\frac{\theta-1}{2(1+\theta x)(1+x)}$, which is negative for all $0<\theta<1$, and therefore $h_{\theta}(x)$ is decreasing in $x$ and does not converge to 0. We then show that the sufficient conditions of Theorem 2 are satisfied: $h(\theta)$ exists, and is equal to $\lim_{x\rightarrow\infty}\frac{1}{2}\log\frac{1+\theta x}{1+x}=\frac{1}{2}\log(\theta)$; $r=1-q$ and hence the gap $\alpha$ is finite. Furthermore, [1] showed that minimizing $\alpha$ over all $q$ gives a constant additive gap, independent of $q$, that is equal to $0.72$.

Next, we note that all bounded utility functions belong to class $(B)$. These are functions $u$ where there exists some constant $M<\infty$ such that $u(x)\leq M,~{}\forall x$. Examples for these include: $u(x)=1-e^{-\beta x}$ for some $\beta>0$, and $u(x)=x/(1+x)$. To see that these functions belong to class $(B)$, observe that $\lim_{x\rightarrow\infty}u(x)=M$ by monotonicity of $u$, and hence $\lim_{x\rightarrow\infty}u(\theta x)-u(x)=0$. We also note that class $(B)$ is not only inclusive of bounded utility functions. For example, the unbounded function $u(x)=\sqrt{\log(1+x)}$ satisfies $\lim_{x\rightarrow\infty}\sqrt{\log(1+\theta x)}-\sqrt{\log(1+x)}=0$ and therefore belongs to class $(B)$. For such unbounded functions in class $(B)$, the FFP is not only within a constant additive gap of the optimal solution, it is asymptotically optimal as well, as indicated by Theorem 3.

Note that one can find a (strict) lower bound on $h(\theta)$ for some utility functions if it allows more plausible computation of $\alpha$, or if $h(\theta)$ itself is not direct to compute. For instance, for any bounded utility function $u$, the following holds: $h(\theta)\geq(\theta-1)M$, where $M$ is the upper bound on $u$. To see this, observe that by concavity of $u$ and the fact that $u(0)=0$ we have $\inf_{x}u(\theta x)-u(x)\geq(\theta-1)\sup_{x}u(x)$. This gives $\alpha\geq\sum_{t=0}^{\infty}q(1-q)^{t}\left((1-q)^{t}-1\right)M$, which is no smaller than $-\frac{1}{2}M$ if we further minimize over $q$. Another example is $u(x)=\frac{1}{2}\log\left(1+\sqrt{x}\right)$, which belongs to class $(A)$. We observe that $h(\theta)$ in this case is lower bounded by $\frac{1}{2}\log(\theta)$. Hence, this function admits an additive gap no larger than $0.72$ calculated in [1] for $u(x)=\frac{1}{2}\log(1+x)$.

Finally, we note that the conditions of Theorem 2 are only sufficient for the FFP defined in (5) to be within an additive gap from optimal. For instance, consider $u(x)=\sqrt{x}$. This function belongs to class $(A)$ as $h_{\theta}(x)$ does not converge to 0. In fact, $h_{\theta}(x)$ is unbounded below and $h(\theta)$ does not exist. This means that any FFP of the form $\tilde{g}_{t}=\theta b_{t}$, for any choice of $0<\theta<1$, is not within a constant additive gap from the upper bound $\sqrt{\mu}$. However, there exists another FFP (with a different fraction than $q$ in (4)) that is optimal in the case of Bernoulli arrivals. Since $u^{\prime}(0)=\infty$, we use (21) to find the optimal $\lambda$, where $f(x)=1/(4x^{2})$, and substitute in (19) to get that the optimal transmission scheme is fractional: $g_{t}=\hat{p}\left(1-\hat{p}\right)^{(t-1)}B,~{}\forall t$, where the transmitted fraction $\hat{p}\triangleq 1-(1-p)^{2}$. This shows that one can pursue near optimality results under an FFP by further optimizing the fraction of power used in each time slot, and comparing the performance directly to the optimal solution instead of an upper bound. While in this work, we compared the lower bound achieved by the FFP to a universal upper bound that works for all i.i.d. energy arrivals.