The Optimal Power Control Policy for an Energy Harvesting System with Look-Ahead: Bernoulli Energy Arrivals

TL;DR

This paper derives the optimal power control policy for an energy harvesting communication system with Bernoulli energy arrivals, finite battery, and look-ahead capability, maximizing long-term throughput.

Contribution

It provides a complete characterization of the optimal power control policy considering look-ahead and Bernoulli arrivals, advancing energy harvesting communication strategies.

Findings

Optimal policy maximizes long-term average throughput.

Look-ahead improves power control decisions.

Characterization applies to finite battery and Bernoulli arrivals.

Abstract

We study power control for an energy harvesting communication system with independent and identically distributed Bernoulli energy arrivals. It is assumed that the transmitter is equipped with a finite-sized rechargeable battery and is able to look ahead to observe a fixed number of future arrivals. A complete characterization is provided for the optimal power control policy that achieves the maximum long-term average throughput over an additive white Gaussian noise channel.

Equations

$$P_{E}(e)=\left\{\begin{array}[]{ll}p&:e=B,\\ 1-p&:e=0.\end{array}\right.$$

$$\begin{array}[]{c}A_{\tau}=\mathcal{A}_{\tau}({(E_{t})_{t=1}^{\tau+w}},B_{1})\>,\\ \text{subject to $A_{\tau}\leq B_{\tau}$}\end{array}$$

$$B_{\tau }=\min \{B_{\tau -1}-A_{\tau -1}+E_{\tau },B\}.$$

$${\Gamma }_T^{\pi (w)}\triangleq \frac{1}{T}\mathbb{E}\left(\sum _{\tau =1}^T\frac{1}{2}\log (1+\gamma {\mathcal{A}}_{\tau }((E_t{)}_{t=1}^{\tau +w},B_1))\right),$$

$${\Gamma }_{B_1}^{\ast }\triangleq \underset{\pi (w)}{\sup }\underset{T\to \infty }{\mathrm{liminf}}{\Gamma }_T^{\pi (w)}.$$

$$S_{\tau }=(B_{\tau },E_{\tau +1},E_{\tau +2},\dots ,E_{\tau +w}).$$

$$\begin{array}[]{c}A_{\tau}=\mathcal{A}(S_{\tau})\>.\\ \text{subject to $A_{\tau}\leq B_{\tau}$}\end{array}$$

$$D_{\tau}\triangleq\left\{\begin{array}[]{ll}0\quad\text{: if }E_{\tau+t}=0\text{ for all}\>t\in\{1,\ldots,w\},\\ \min\{t:1\leq t\leq w,E_{t+\tau}=B\}\quad\text{: O.W.}\>.\end{array}\right.$$

$$a_{\tau}=\mathcal{A}(b_{\tau},e_{\tau+1},\ldots,e_{\tau+w})=\left\{\begin{array}[]{ll}\frac{b_{\tau}}{d_{\tau}}&\text{: }d_{\tau}\neq 0\>,\\ \mathcal{A}(b_{\tau},0,\cdots,0)&\text{: O.W.}\>.\end{array}\right.$$

$$S_{\tau }=(B_{\tau },D_{\tau }).$$

$${\xi }_j^{\ast }\triangleq \mathcal{A}(b_j,0,\cdots ,0):j\in \text{mathds}N,$$

$$\mathbb{E}(L)=\frac{1}{p}.$$

$$\underset{T\to \infty }{\mathrm{liminf}}{\Gamma }_T^{\pi (w)}$$

$${\mathcal{T}}_{\infty }(\{x_j{\}}_{i=1}^{\infty })\triangleq$$

$$+\sum _{j=1}^{\infty }p(1-p{)}^{j+w-1}\frac{1}{2}\log (1+\gamma x_j)$$

$$+\sum _{k=1}^{\infty }$$

$${\Gamma }^{\ast }={\mathcal{T}}_{\infty }(\{{\xi }_i^{\ast }{\}}_{i=1}^{\infty }).$$

$$Q_j=\frac{B}{k}\text{for }j\in \{1,\dots ,k\}.$$

$$Q_{j}=\left\{\begin{array}[]{ll}\xi_{j}^{*}&:j\in\{1,\ldots,k-w\},\\ \frac{B-\sum_{j=1}^{k-w}\xi_{j}^{*}}{w}&:j\in\{k-w+1,\ldots,w\}\>.\end{array}\right.$$

$${\Gamma }^{\ast }$$

$${\Gamma }^{\ast }={\mathcal{T}}_{\infty }^{\ast }.$$

$$\frac{1-p}{1+\gamma {\xi }_{j+1}^{\ast }}$$

$$\sum _{j=1}^{\infty }{\xi }_j^{\ast }$$

$${\xi }_j^{\ast }<\frac{B-{\sum }_{i=1}^j{\xi }_i^{\ast }}{w}.$$

$${\Gamma }^{\ast }=\sum _{k=1}^{\infty }{p}^2(1-p{)}^{k-1}\frac{k}{2}\log (1+\gamma \frac{B}{k}).$$

$${\mathcal{T}}_N({\xi }_1,\cdots ,{\xi }_N)\triangleq {\mathcal{T}}_{\infty }({\xi }_1,\cdots ,{\xi }_N,0,0,\cdots )$$

$$=\sum _{k=1}^w{p}^2(1-p{)}^{k-1}\frac{k}{2}\log (1+\gamma \frac{B}{k})$$

$$+\sum _{j=1}^{N}p(1-p{)}^{j+w-1}\frac{1}{2}\log (1+\gamma {\xi }_j)$$

$$+\sum _{k=1}^N{p}^2(1-p{)}^{k+w-1}\frac{w}{2}\log (1+\gamma \frac{B-{\sum }_{j=1}^k{\xi }_j}{w})$$

$$+p(1-p{)}^{w+N}\frac{w}{2}\log (1+\gamma \frac{B-{\sum }_{j=1}^N{\xi }_j}{w}).$$

$${\mathcal{T}}_N^{\ast }=\sup {\mathcal{T}}_N({\xi }_1^{(N)},\cdots ,{\xi }_N^{(N)}),$$

$${\xi }_j^{(N)}\ge 0,$$

$$\sum _{j=1}^N{\xi }_j^{(N)}\le B.$$

$${\mathcal{T}}_N^{\ast }({\xi }_1^{(N)\ast },\cdots ,{\xi }_N^{(N)\ast })$$

Full text

I Introduction

Supplying required energy of a communication system by energy harvesting (EH) from natural energy resources is not only beneficial from the environmental perspective, but also essential for long-lasting self-sustainable affordable telecommunication which can be employed in places with no electricity infrastructure. On the other hand, the EH systems require to handle related challenges, such as varying nature of green energy resources and limited battery storage capacity.

Consider an EH communication system with a transmitter (TX) and a receiver (RX), connected by an additive white Gaussian noise (AWGN) channel. The TX is equipped with a rechargeable battery of a given storage size, and is capable of harvesting the energy arrivals, which are assumed to be independent and identically distributed (i.i.d.). The communication session consists of $T$ (discrete) time slots, and the instantaneous rate achieved at each time slot is a function of its allocated energy. A power control policy specifies the energy assignment across the time slots according to the initial battery energy level, the harvested energy sequence thus far, and the knowledge of future harvested energy arrivals. The reward associated with each policy is the average throughput over $T-$horizon. The average throughput optimization (ATO) problem aims to determine the optimal policy that achieves the maximum average throughput.

In the seminal paper [1], the authors studied the ATO problem over a finite horizon for the offline model, where the energy arrivals are non-causally known at the TX, and they derived the optimal policy for this model based on  [2]. The analyses of the offline model for more general channels can be found in [3, 4, 5, 6, 7] (and the references therein). In general, the optimal policies for the offline model strive to allocate the energy across the time horizon as uniformly as possible while trying to avoid energy loss due to the battery overflow.

In landmark paper [8], the authors studied the ATO problem over an infinite horizon for the online model, where the energy arrivals are causally known at the TX. They determined the optimal policy for Bernoulli energy arrivals and established the approximate optimality of the fixed fraction policy for general energy arrivals. Similar results were derived in  [9, 10] for a general concave and monotonically increasing utility function. In papers [11, 12], the authors studied the same problem with unlimited size battery and developed three simple online optimal policies which are optimal for the offline case as well.

In this paper, we study the setup where the TX is able to look ahead to observe a window of size $w$ of future energy arrivals. In fact, the online model and offline model correspond to the extreme cases $w=0$ and $w=\infty$, respectively. Therefore, our formulation provides a link between these two models, which have been largely studied in isolation. From the practical perspective, the TX often has a good estimation of the amount of available energy in near future, because such energy is already harvested but not yet converted to a usable form. We investigate the ATO problem over an infinite horizon for this new setup. Specifically, we focus on Bernoulli energy arrivals and characterize the corresponding optimal policy. The main difference between this optimal policy and that of [8, 9, 10] is as follows. For the new policy, if no energy arrival is seen in the look-ahead window, the battery always keeps some energy for future and spends a portion of available energy in the current time slot. In contrast, for the policy in [8, 9, 10], the energy is only allocated to a fixed number of time slots after each battery charge and no energy is expended beyond that as the battery becomes depleted.

The organization of this paper is as follows. In Section II, we introduce the model and problem. In Section III, we develop the structure of an optimal policy and justify the problem solving strategy. In Section IV, we establish the main results and completely characterize the optimal policy. In Section V, we finally conclude this paper.

II Problem Definitions

The notations of this paper are as follows. $\mathds{N}$ and $\mathds{R}^{+}$ represent the set of natural numbers and set of positive real numbers, respectively. Random variables are denoted by capital letters and their realizations are written in lower case letters. The functions are denoted by calligraphic font. $\mathbb{E}$ is reserved for the expectation. The logarithms are in base 2.

Consider a point-to-point AWGN quasi-static fading channel from a TX to an RX, where the channel gain is a constant $\gamma\in\mathds{R}^{+}$ for the entire communication session. The communication is discrete time with time slot $\tau\in\mathds{N}$ as formulated by $Y_{\tau}=\sqrt{\gamma}X_{\tau}+Z_{\tau}$, where $X_{\tau}$ and $Y_{\tau}$ are the transmitted signal and the received signal, respectively; $Z_{\tau}$ is the Gaussian noise with zero mean and unit variance. The TX is capable of harvesting energy from the environment and is equipped with a rechargeable battery with finite size $B>0$. The exogenous (harvested) energy arrivals are assumed to be i.i.d. process with known marginal distribution $P_{E}$. In this work, unless specified otherwise, we assume that $P_{E}$ is Bernoulli$(p)$-$\{0,B\}$, where $0<p<1$, defined as

[TABLE]

Denote the energy level stored in the battery by random process $\{B_{\tau},\tau\in\mathds{N}\}$ with initial level, $B_{1}=\beta$, where $0\leq\beta\leq B$. If the energy arrives at some time instant $\tau\in\mathds{N}$, $E_{\tau}=B$, and the battery is fully charged to $B_{\tau}=B$. In this case, any remained energy in the battery is overflowed and wasted away. If no energy arrives, $E_{\tau}=0$, and the battery energy level is not escalated at time slot $\tau$.

In this paper, we assume that the TX is able to look ahead with a fixed window size $w\in\mathds{N}$: the realization of the energy arrival sequence $\{E_{t}\}_{t=1}^{\tau+w}$ is known to the TX at time $\tau$.

The TX sends energy $A_{\tau}$ as an action at time slot $\tau$, where the energy is determined by a (randomized) action function

[TABLE]

and gain throughput $\mathcal{R}_{\tau}(A_{\tau})=\frac{1}{2}\log(1+\gamma A_{\tau})$, as the reward of time $\tau$. Then, the battery energy level becomes

[TABLE]

A look-ahead policy $\pi(w)$ is characterized by sequence of action functions $\{\mathcal{A}_{\tau}\}_{\tau=1}^{\infty}$. For a fixed communication session time $T\in\mathds{N}$, $\pi(w)$ gains the average (expected) throughput over $T$-horizon

[TABLE]

as its associated reward, where the expectation is over all energy arrival sequences $\{e_{t}\}_{t=1}^{T}$.

Definition 1**.**

The largest average reward (channel throughput) over infinite horizon (long term) is defined as

[TABLE]

If $\Gamma^{*}_{B_{1}}$ is attainable by a policy $\pi^{*}(w)$, it is called optimal.

Remark 1*.*

It can be shown [8, Appendix B] that $\Gamma^{*}_{B_{1}}$ does not depend on $B_{1}$. Hence, we can drop the subscript $B_{1}$ in (5), and assume $B_{1}=B$ without loss of generality (WLOG).

In this paper, we seek $\Gamma^{*}$ and the corresponding optimal policy $\pi^{*}(w)$ according to Definition 1.

III Problem Solving Strategy

First, assume that the distribution of the harvested energy, $P_{E}$, is arbitrary. In general, $A_{\tau}$ depends on $(B_{1},\{E_{t}\}_{t=1}^{\tau+w})$. According to [13, 14], there is no loss of optimality in (5) if the supremum is taken over deterministic Markovian stationary policies which only rely on system state

[TABLE]

Indeed, (5) is attainable by an optimal stationary policy $\pi^{*}(w)$. Given $S_{\tau}$ with finite length $w+1$, knowing energy arrivals $\{E_{t}\}_{t=1}^{\tau}$ does not enhance $\Gamma^{*}$. Note that the action $A_{\tau}$ is not only determined by the current energy level $B_{\tau}$, but also it can be affected by the observed future energy arrivals within the look ahead window. As the optimal policy is Markovian and stationary, the action function (2) can be simplified to the time-invariant function

[TABLE]

Now, focus on Bernoulli distribution as defined in (1). In this case, the state (6) can be simplified as follows. Let random variable $D_{\tau}$ be the time distance of the earliest energy arrival located inside the look-ahead window. Specifically, define

[TABLE]

For any given energy arrival sequence $\{e_{t}\}_{t=1}^{\tau+w}$ and battery level $b_{\tau}$ at time $\tau\in\mathds{N}$, if an energy arrival is observed at time $\tau$, i.e., $d_{\tau}>0$, then the optimal policy uniformly assigns instant battery energy $b_{\tau}$ to the following $d_{\tau}$ time spots. This is due to the concavity of the reward function (4). Otherwise, $\mathcal{A}(b_{\tau},0,\cdots,0)$ is the action at time $\tau$, which will be determined in the sequel. Therefore, the action function (7) of the stationary optimal policy $\pi^{*}(w)$ is given by

[TABLE]

From (8), we conclude that $a_{\tau}$ (and so the associated reward) can be uniquely determined by $(b_{\tau},d_{\tau})$. Hence, the system state for Bernoulli energy arrival can be reduced to

[TABLE]

A non-negative sequence $\{x_{j}\}_{j=1}^{N}$ with length $N\in\mathds{N}$ is called admissible, if $\sum_{j=1}^{N}x_{j}\leq B$. Let $b_{1}=B$. Define admissible sequence $\{\xi^{*}_{j}\}_{j=1}^{\infty}$ associated with $\pi^{*}(w)$ by

[TABLE]

where $b_{j+1}=b_{j}-\xi^{*}_{j}$. Due to (8), if the battery is charged up at some time $\tau=\tau_{1}$, ($b_{\tau_{1}}=B$), but no arrival occurs later, ($e_{\tau}=0$  : $\tau>\tau_{1}$), $\pi^{*}(w)$ sends $a_{\tau}=\xi^{*}_{\tau-\tau_{1}+1}$ for $\tau\geq\tau_{1}$. In the sequel, $\{\xi_{i}^{*}\}_{i=1}^{\infty}$ and its properties are investigated. Once $\{\xi_{i}^{*}\}_{i=1}^{\infty}$ is determined, $\pi^{*}(w)$ follows Algorithm 1.

The times in interval $[\tau_{1},\tau_{2})$ with property $E_{\tau_{1}}=E_{\tau_{2}}=B$ and $E_{t}=0$ for $\tau_{1}<t<\tau_{2}$ is called a cycle with start time $\tau_{1}$. Let define random variable $L=\tau_{2}-\tau_{1}$ as the (duration) time of the cycle. Due to (1), the distribution of $L$ is Geometric, i.e., $Pr\{L=k\}=p(1-p)^{k-1}$ for $k\in\mathds{N}$, with mean

[TABLE]

If a stationary policy is employed, the processes of $\{B_{\tau},\tau\in\mathds{N}\}$, $\{S_{\tau},\tau\in\mathds{N}\}$, and $\{A_{\tau},\tau\in\mathds{N}\}$ are non-delayed regenerative processes [15, Section 7.5] with cycles of time $L$: when the battery charges up to $B$ at some cycle start time $\tau_{1}$, memories of the processes are reset. Consequently, $\{(B_{t},S_{t},A_{t})\}_{t=\tau_{1}}^{\infty}$ does not statistically depend on $(\{B_{t}\}_{t=1}^{\tau_{1}-1},\{S_{t}\}_{t=1}^{\tau_{1}-1},\{A_{t}\}_{t=1}^{\tau_{1}-1})$ and time $\tau_{1}$.

As $E(L)$ in (11) and $|\mathcal{R}_{\tau}(A_{\tau})|$ are bounded and $\{A_{\tau},\tau\in\mathds{N}\}$ is a non-delayed regenerative process, the “renewal reward theorem” [15, Section 7.4] can be utilized to simplify the long-term average throughput achieved by a policy $\pi(w)$.

[TABLE]

where (12) is due to (11) and $Q_{j}$ is the energy assigned to time $j^{th}$ of a cycle conditioned on $k$.The following definition is helpful to calculate (12).

Definition 2**.**

Let $\{x_{j}\}_{j=1}^{\infty}$ be an admissible sequence. Define

[TABLE]

Lemma 1**.**

The long-term average throughput (4) of optimal policy $\pi^{*}(w)$ with associated sequence $\{\xi_{i}^{*}\}_{i=1}^{\infty}$ satisfies

[TABLE]

Proof.

The proof follows from Definition (1), (12) and Definition 2. First, energy assignments $Q_{j}$ in (12) for $\pi^{*}(w)$ is determined in the following. Assume a new cycle is started at time $\tau=\tau_{1}$ and so the battery energy level is $B$. Given the cycle time $L=k$, the following two cases can be considered.

1. a.

Case $k\leq w$: In this case, $D_{\tau}=k$ as the arrival is observed in the window. Hence, according to (8), we have

[TABLE] 2. b.

Case $k>w$: In this case, the TX allocates $\{\xi_{j}^{*}\}_{j=1}^{k-w}$ to the first $k-w$ time slots of each cycle; then, it uniformly distributes the remained energy of the battery $(B-\sum_{i=1}^{k-w}Q_{i})$ into time slots $k-w+1,\ldots,k$ due to (8), as soon as the arrival is observable (look-ahead window covers time $k$).

[TABLE]

Finally, the proof can be concluded from the following calculation of (12) for $\pi^{*}(w)$ based on (15) and (16).

[TABLE]

∎

IV Properties of the Optimal Policy

In this section, we characterize the energy sequence $\{\xi_{j}^{*}\}_{j=1}^{\infty}$ and its properties. The main result of this paper is as follows.

Theorem 1**.**

Define $\mathcal{T}_{\infty}^{*}=\sup\mathcal{T}_{\infty}(x_{1},x_{2},\cdots)$, where the supremum is over all admissible sequences $\{x_{j}\}_{j=1}^{\infty}$. Then, the maximum long-term average reward (channel throughput) of the look-ahead model is given by

[TABLE]

Moreover, $\{\xi_{j}^{*}\}_{j=1}^{\infty}$ induced by the optimal policy is the unique maximizer of $\mathcal{T}_{\infty}$ and is the unique sequence satisfying

[TABLE]

It is also a strictly decreasing positive sequence with property

[TABLE]

Remark 2*.*

The optimal average throughput of the studied model coincides with the optimal average throughput of the non-causal model if $w=\infty$ is set in (17). In this case the optimal average throughput is given by

[TABLE]

In Fig. 1, we illustrate the (long-term) average throughput $\Gamma^{*}$ as a function of window size $w$ using (18) for the given system parameters. Although the optimal throughput rate is an increasing function of $w$, if the window size $w\geq 5$ in Fig. 1, this communication system achieves the optimal average throughput corresponds to $w=\infty$ within a gap smaller than $0.5\%$. The rest of this section is devoted to the proof of theorem 1.

The proof of (18) is as follows. As mentioned, there exists a stationary policy $\pi^{*}(w)$ which attains $\Gamma^{*}$ [13, 14]. That policy also satisfies (14) in which $\{\xi_{j}^{*}\}_{j=1}^{\infty}$ is associated with $\pi^{*}(w)$. As $\pi^{*}(w)$ is optimal, $\{\xi_{j}^{*}\}_{j=1}^{\infty}$ must maximize $\mathcal{T}_{\infty}$ in (14) over all admissible sequences. As $\mathcal{T}_{\infty}$ is a strictly concave function, the maximizing sequence is unique. The unique maximizer $\{\xi_{j}^{*}\}_{j=1}^{\infty}$ indeed achieves $\mathcal{T}_{\infty}^{*}$. Hence, (18) is justified. To investigate sequence $\{\xi_{j}^{*}\}_{j=1}^{\infty}$ as the unique maximizer of $\mathcal{T}_{\infty}$, we need to define the corresponding $N-$dimensional optimization problem in the following subsection to employ Karush-Kuhn-Tucker (KKT) conditions. Then, we investigate the relation between the finite dimensional optimization problem and the infinite dimensional optimization problem.

IV-A $N-$Dimensional Optimization Problem.

Let define the following $N-$dimensional optimization problem based on Definition 2.

Definition 3**.**

Fix $N\in\mathds{N}$. Recall Definition 2, and set $x_{j}=\xi_{j}$ for $j\in\{1,\ldots,N\}$ and $x_{j}=0$ for $j>N$. Define

[TABLE]

The $N-$dimensional optimization problem is defined as

[TABLE]

where the supremum is over all sequences $\{\xi_{j}^{(N)}\}_{j=1}^{N}$ subject to

[TABLE]

The maximizer of $\mathcal{T}_{N}$, if exists, is denoted by $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$.

Lemma 2**.**

The following statements are valid for $\mathcal{T}_{N}^{*}$.

1. (a).

There exists a unique maximizer $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ of $\mathcal{T}_{N}$. 2. (b).

$\mathcal{T}_{N}^{*}$ is an increasing function of $N$. 3. (c).

$\lim\limits_{N\rightarrow\infty}\mathcal{T}_{N}^{*}=\mathcal{T}_{\infty}^{*}$ and it is finite.

Proof.

(a) follows the fact that function $\mathcal{T}_{N}$ is a continuous bounded concave function defined on a compact set. (b) is due to the fact that domain of $\mathcal{T}_{N}$ is a subset of domain of $\mathcal{T}_{N+1}$. (c) follows from the fact that $\mathcal{T}_{N}$ is a continuous bounded function and is increasing, i.e., $\mathcal{T}_{N}^{*}\leq\mathcal{T}_{\infty}^{*}$, and thus $\lim\limits_{N\rightarrow\infty}\mathcal{T}_{N}^{*}$ exists and it is finite. Indeed, the following inequality proves that this limit is $\mathcal{T}_{\infty}^{*}$ due to the squeeze theorem.

[TABLE]

where $\epsilon_{N}\triangleq\frac{(1-p)^{w+N}}{2}\left[\log(1+\gamma B)+pw\log(1+\frac{\gamma}{w}B)\right]$ is a positive number with property $\lim\limits_{N\rightarrow\infty}\epsilon_{N}=0$. ∎

In Corollary 2 in the sequel, we will prove that $\mathcal{T}_{N}^{*}$ is indeed a strictly increasing function, which is stronger than Lemma 2-part(B).

The $N$-dimensional optimization problem can be solved by the KKT method. A necessary and sufficient condition for sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ to attain $\mathcal{T}^{*}_{N}$ is to satisfy

[TABLE]

where $j\in\{1,2,\cdots,N\}$, and $\lambda^{(N)},\{\mu^{(N)}_{i}\}_{i=1}^{N}$ are non-negative real numbers.

IV-B Properties of Sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$.

In this subsection, we investigate the properties of the sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ which achieves $\mathcal{T}_{N}^{*}$ based on the KKT conditions (24).

Definition 4**.**

The effective length of an admissible energy sequence is the largest (time) index beyond which the rest of the sequence vanishes. Specifically, $N_{eff}$ is called the effective length of $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ if

[TABLE]

Lemma 3**.**

The effective length of sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ is $N$.

Proof.

First, we show that there exists at least one non-zero elements in the sequence of $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$. Assume that this claim is not valid, and an all zero sequence is the optimal solution which satisfies the KKT conditions (24). Thus,

[TABLE]

Hence, $\lambda^{(N)}=0$ due to (24c). From $\lambda^{(N)}=0$, $\mu_{N}\geq 0$, (24a) (when $j=N$ is set) and (28), we should have

[TABLE]

However, this equation is only valid when $\xi_{N}\geq\frac{\gamma}{w}B>0$ due to $B>0$. This inequality contradicts with (27). Consequently, the largest (time) index of the non-zero element ($J$) exists with property (25).

Second, we prove that $J<N$ is not valid in the following. Suppose $J<N$. Define $B^{\prime}=\sum_{j=1}^{J}\xi_{j}^{(N)*}$. Note that we have

[TABLE]

where the second inequality is due to $\xi_{J}^{(N)*}>0$ and (24b). First, assume $B=B^{\prime}$; Setting $j=J$ in (24a) gives

[TABLE]

which is contradictory with the fact that $\lambda^{(N)}\geq 0$. Next consider the case $B^{\prime}<B$. In this case, $\lambda^{(N)}=0$. Setting $j=J+1$ in (24a) gives

[TABLE]

However, in (32), the bracket is strictly positive. Hence, the left hand side of (32) is strictly positive. This lead to a contradiction. Therefore, $J=N$ always holds. ∎

Corollary 1**.**

The last element of the optimal sequence $\{\xi_{i}^{(N)*}\}_{i=1}^{N}$ is always non-zero, i.e., $\xi_{N}^{(N)*}>0$ and coefficient $\mu^{(N)}_{N}=0$.

Proof.

Due to Lemma 3, the largest non-zero element is the $N^{th}$ element of the sequence. $\mu^{(N)}_{N}=0$ is due to (24b). ∎

Corollary 2**.**

$\mathcal{T}_{N}^{*}$ is a strictly increasing function of $N$.

Proof.

[TABLE]

where the last inequality is due to the fact that $(\xi_{1}^{(N)*},\cdots,\xi_{N}^{(N)*},0)$ can not be the unique maximizer of $\mathcal{T}_{N+1}$, because $\xi_{N+1}^{(N+1)*}>0$ due to Corollary 1. ∎

Lemma 4**.**

For any $j\in\{1,2,\cdots,N-1\}$, we have

[TABLE]

where inequality holds in the strict sense for $j<N$.

Proof.

If $\xi_{j}^{(N)*}=0$, the inequality holds because $\sum_{i=1}^{j}\xi_{i}^{(N)*}<B$ according to (22b) and Corollary 1 when $j<N$. Otherwise, if $\xi_{j}^{(N)*}>0$, then $\mu^{(N)}_{j}=0$ due to (24b). Hence, we can derive the expressions (34)-(37) at the bottom of this page, where (34) is due to (24a) and $\lambda^{(N)}\geq 0$, (35) follows from $B-\sum_{i=1}^{k-w}\xi_{i}^{(N)*}\leq B-\sum_{i=1}^{j}\xi_{i}^{(N)*}$, for $k=w+j+1,\cdots,w+N-1$, due to (22b), (36) holds strictly only if $j<N$, because $B-\sum_{i=1}^{N}\xi_{i}^{(N)*}<B-\sum_{i=1}^{j}\xi_{i}^{(N)*}$ due to Corollary 1 for $j<N$. Therefore, the lemma is concluded from (37). ∎

For any fixed $N$, let define parameter

[TABLE]

From Corollary 1 and Lemma 4, we conclude that

[TABLE]

Also, from (24c) and (38), we conclude that

[TABLE]

In the following lemma, we investigate the behaviour of sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ as a function of time index $j$ when $N$ is fixed.

Lemma 5**.**

$\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ is a strictly decreasing positive sequence.

Proof.

From (24a), for two successive terms $j<N$ and $j+1$, we have

[TABLE]

and

[TABLE]

respectively. Subtracting the first equation from the second one, we obtain

[TABLE]

From Lemma 4 (for case $j<N$), we have $\frac{\frac{\gamma}{2}}{1+\gamma\frac{B-\sum_{i=1}^{j}\xi_{i}^{(N)*}}{w}}<\frac{\frac{\gamma}{2}}{1+\gamma\xi_{j}^{(N)*}}$. Hence, (LABEL:eq:successive_terms-0) can be simplified into

[TABLE]

First, note that $\xi_{N}^{(N)*}>0$ due to Corollary 1. Second, start with $j=N-1$. In this case $\mu^{(N)}_{j}\geq 0$ and $\mu^{(N)}_{j+1}=0$ due to Corollary 1. Hence, from (41), we conclude that $\xi_{j+1}^{(N)*}<\xi_{j}^{(N)*}$. Therefore, $\xi_{j}^{(N)*}>0$ due to Corollary 1, and thus $\mu^{(N)}_{j}=0$ due to (24b). Then, repeat this justification for $j=N-2,N-3,\ldots,1$ in the descending order to establish this lemma for all $j\in\{1,\cdots,N-1\}$. ∎

Remark 3*.*

Note that Lemma 5 is different from Lemma 3 regarding positivity property of each element. This lemma only guarantees that a finite number of zero elements in sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ exists, while Lemma 5 guarantees no zero element exists and thus it is stronger. As a result of Lemma 5, it can be concluded that

[TABLE]

holds for any $j\in\{1,2,\ldots,N\}$ in (24b).

In the following lemma, we establish a recursive expression to simplify the KKT conditions and to obtain energy sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$, recursively.

Lemma 6**.**

The following identities hold for energy sequence $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$, which satisfies the KKT conditions (24).

[TABLE]

Proof.

The KKT condition (24a) can be simplified due to (39) and (42). For $j=N$, we have

[TABLE]

from which (43a) is derived. For $j\in\{1,2,\ldots,N-1\}$, (43b) is derived from (LABEL:eq:successive_terms-0) and (42).

∎

Remark 4*.*

From Lemma 6, we conclude that the equality in (4) for $j=N$ always holds.

In the following lemma, we investigate the behaviour of sequence $\{\xi_{j}^{(N)*}\}_{N=j}^{\infty}$ as a function of $N$ for a fixed time instant $j$.

Lemma 7**.**

Consider sequence $\{\xi_{j}^{(N)*}\}_{N=j}^{\infty}$ for a given fixed $j\in\mathds{N}$. Then, $\{\xi_{j}^{(N)*}\}_{N=j}^{\infty}$ is a strictly decreasing function of $N$.

Proof.

Let consider two different assumptions for $\xi_{1}^{(N)*}$ and $\xi_{1}^{(N+1)*}$. First, assume that $\xi_{1}^{(N+1)*}\geq\xi_{1}^{(N)*}$. If this is the case, from (43b), we can deduce

[TABLE]

However, this conclusion contradicts the following inequality for $j=N$.

[TABLE]

where (45) follows from (33) (of Lemma 4), (46) follows from (44), and (47) follows from (43a). Hence, the assumption $\xi_{1}^{(N+1)*}<\xi_{1}^{(N)*}$ is correct. Therefore, from (43b), we can deduce

[TABLE]

∎

Lemma 8**.**

Sequence $\{B^{(N)}\}_{N=1}^{\infty}$ is a strictly decreasing function of $N$.

Proof.

From Lemma 7, we conclude that $\xi_{N}^{(N+1)*}<\xi_{N}^{(N)*}$. On the other hand, from Lemma 5, we conclude that $\xi_{N+1}^{(N+1)*}<\xi_{N}^{(N+1)*}$. Hence, inequality $\xi_{N+1}^{(N+1)*}<\xi_{N}^{(N)*}$ leads to inequality $B^{(N+1)}<B^{(N)}$ due to (43a). ∎

Corollary 3**.**

The limit of sequence $\{B^{(N)}\}_{N=1}^{\infty}$ exists as follows.

[TABLE]

Proof.

Sequence $\{B^{(N)}\}_{N=1}^{\infty}$ is a strictly decreasing function of $N$, due to Lemma 8, and positive due to (38). Hence, it converges to a non-negative real number. To find the limit, we can bound $B^{(N)}$ as follows.

[TABLE]

where (49) follows from (43a), (50) follows from Lemma 5, and (51) follows from energy constraint (22b). The limit is concluded from (51). ∎

IV-C Properties of sequence $\{\xi_{j}^{*}\}_{j=1}^{\infty}$.

Lemma 9**.**

The limits of sequence $\{\xi_{j}^{(N)*}\}_{N=j}^{\infty}$ for any $j\in\mathds{N}$, where $j$ is fixed, exists as follows.

[TABLE]

where $\xi_{j}^{*}$ is element $j^{th}$ of the maximizing sequence of $\mathcal{T}_{\infty}$.

Proof.

Sequence $\{\xi_{j}^{(N)*}\}_{N=j}^{\infty}$, for any fixed $j\in\mathds{N}$, consists of positive elements due to Lemma (5) and it strictly descends as a function of $N$ due to Lemma 7. Hence, for each $j\in\mathds{N}$, there exits a finite number $\alpha_{j}$ such that

[TABLE]

In the following, we prove that $\{\alpha_{j}\}_{j=1}^{\infty}$ is the unique maximizer of $\mathcal{T}_{\infty}$. For any fixed $N_{0}$, we have

[TABLE]

where (54) follows from Lemma 2-(part C) and (53), 55 follows from Lemma 2-(part B), (56) follows from (23). Now, if $N_{0}\rightarrow\infty$, then $\epsilon_{N_{0}}\rightarrow 0$ due to (23), and thus (56) contradicts with Lemma 2-(part A) unless $\alpha_{j}=\xi_{j}^{*}$, or equivalently,

[TABLE]

∎

Eventually, in Theorem 1, (19a) follows directly from (43b) (of Lemma 6) and (57) (of Lemma 9). Also, the energy constraint (19b) follows from Corollary 3 and (57). Also, (20) follows from Lemma 4 and (57). The fact that $\{\xi_{j}^{*}\}_{j=1}^{N}$ is a strictly decreasing sequence follows from

[TABLE]

where (58) follows from (19a), and (59) follows from (20). This monotone property also approves the positivity of sequence $\{\xi_{j}^{*}\}_{j=1}^{N}$.

In Fig. 2, $\{\xi_{j}^{(N)*}\}_{j=1}^{N}$ (following by trailing zeros) is sketched over time for different values of $N$, when the parameters of the system are as given. This figure numerically illustrates the results of this section.

Remark 5*.*

After an energy arrival, the optimal policy $\pi^{*}(w)$ assigns a strictly positive energy sequence $\{\xi_{j}^{*}\}_{j=1}^{\infty}$ to time slots according to Algorithm 1 until an arrival is observed in the window. That is, as long as no energy arrival is observed in the window, the TX always sends a positive portion of the battery energy at each time slot, but it never exhausts the battery energy. However, the online optimal policies [8, 9, 10] allocate positive energy only to a fixed number of time slots after any energy arrival and then the battery becomes depleted.

V Conclusions

In this paper, we have introduced a new EH system model, where the TX has the knowledge of future energy arrivals in a look-ahead window. This model provides a bridge between the online model and the offline model, which have been previously studied in isolation. A complete characterization of the optimal power control policy is obtained for the new model with Bernoulli energy arrivals, which has notable differences with its counterpart for the online model.