ACES -- Automatic Configuration of Energy Harvesting Sensors with Reinforcement Learning

TL;DR

This paper presents ACES, a reinforcement learning-based system for optimizing energy harvesting sensors in IoT applications, enabling continuous operation and adaptive sensing in variable lighting conditions.

Contribution

The paper introduces a novel reinforcement learning approach for energy management in indoor light-powered sensors, demonstrating its effectiveness through real-world deployment.

Findings

Sensors adapt to lighting conditions and operate continuously.

Achieved 0.1% dead time in a 2-week deployment.

Captured 86% of motion events compared to battery-powered sensors.

Abstract

Internet of Things forms the backbone of modern building applications. Wireless sensors are being increasingly adopted for their flexibility and reduced cost of deployment. However, most wireless sensors are powered by batteries today and large deployments are inhibited by manual battery replacement. Energy harvesting sensors provide an attractive alternative, but they need to provide adequate quality of service to applications given uncertain energy availability. We propose using reinforcement learning to optimize the operation of energy harvesting sensors to maximize sensing quality with available energy. We present our system ACES that uses reinforcement learning for periodic and event-driven sensing indoors with ambient light energy harvesting. Our custom-built board uses a supercapacitor to store energy temporarily, senses light, motion events and relays them using Bluetooth Low Energy. Using simulations and real deployments, we show that our sensor nodes adapt to their lighting conditions and continuously sends measurements and events across nights and weekends. We use deployment data to continually adapt sensing to changing environmental patterns and transfer learning to reduce the training time in real deployments. In our 60 node deployment lasting two weeks, we observe a dead time of 0.1%. The periodic sensors that measure luminosity have a mean sampling period of 90 seconds and the event sensors that detect motion with PIR captured 86% of the events on average compared to a battery-powered node.