Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels

TL;DR

This paper introduces low-complexity clipped affine policies for online power control in energy harvesting wireless channels, combining approximation, reinforcement learning, and lookahead to achieve near-optimal performance with minimal computational effort.

Contribution

It proposes a novel clipped affine policy framework derived from a relative-value function approximation, enhancing online power control with reinforcement learning and lookahead capabilities.

Findings

Achieves near-optimal performance with less than 2% loss compared to the optimal policy.

Outperforms existing methods across various scenarios.

Uses at most five parameters in the RL-based policy.

Abstract

This paper investigates online power control for point-to-point energy harvesting communications over wireless fading channels. A linear-policy-based approximation is derived for the relative-value function in the Bellman equation of the power control problem. This approximation leads to two fundamental power control policies: optimistic and robust clipped affine policies, both taking the form of a clipped affine function of the battery level and the reciprocal of channel signal-to-noise ratio coefficient. They are essentially battery-limited weighted directional waterfilling policies operating between adjacent time slots. By leveraging the relative-value approximation and derived policies, a domain-knowledge-enhanced reinforcement learning (RL) algorithm is proposed for online power control. The proposed approach is further extended to scenarios with energy and/or channel lookahead. Comprehensive simulation results demonstrate that the proposed methods achieve a good balance between computational complexity and optimality. In particular, the robust clipped affine policy (combined with RL, using at most five parameters) outperforms all existing approaches across various scenarios, with less than 2\% performance loss relative to the optimal policy.