Energy Management in a Cooperative Energy Harvesting Wireless Sensor Network

TL;DR

This paper develops an optimal energy management policy for a distributed sensor network with energy harvesting and sharing capabilities, using deep reinforcement learning to outperform existing methods.

Contribution

It introduces a novel distributed energy sharing policy framework using DDPG, improving upon prior single-node or centralized models.

Findings

DDPG-based policies outperform non-sharing algorithms.

Distributed energy sharing enhances network efficiency.

Policies adapt well to dynamic energy harvesting conditions.

Abstract

In this paper, we consider the problem of finding an optimal energy management policy for a network of sensor nodes capable of harvesting their own energy and sharing it with other nodes in the network. We formulate this problem in the discounted cost Markov decision process framework and obtain good energy-sharing policies using the Deep Deterministic Policy Gradient (DDPG) algorithm. Earlier works have attempted to obtain the optimal energy allocation policy for a single sensor and for multiple sensors arranged on a mote with a single centralized energy buffer. Our algorithms, on the other hand, provide optimal policies for a distributed network of sensors individually harvesting energy and capable of sharing energy amongst themselves. Through simulations, we illustrate that the policies obtained by our DDPG algorithm using this enhanced network model outperform algorithms that do not share energy or use a centralized energy buffer in the distributed multi-nodal case.