Revisiting DNN Training for Intermittently-Powered Energy-Harvesting Micro-Computers

TL;DR

This paper presents a dynamic dropout training method for deep neural networks tailored to energy-harvesting micro-computers, improving accuracy and energy efficiency in intermittent power scenarios.

Contribution

It introduces a novel energy-aware dropout technique that adapts to device architecture and energy variability, enhancing DNN performance under power constraints.

Findings

Achieves 6-22% accuracy improvements over existing methods.

Requires less than 5% additional computation.

Effectively integrates energy profiles with training algorithms.

Abstract

The deployment of Deep Neural Networks in energy-constrained environments, such as Energy Harvesting Wireless Sensor Networks, presents unique challenges, primarily due to the intermittent nature of power availability. To address these challenges, this study introduces and evaluates a novel training methodology tailored for DNNs operating within such contexts. In particular, we propose a dynamic dropout technique that adapts to both the architecture of the device and the variability in energy availability inherent in energy harvesting scenarios. Our proposed approach leverages a device model that incorporates specific parameters of the network architecture and the energy harvesting profile to optimize dropout rates dynamically during the training phase. By modulating the network's training process based on predicted energy availability, our method not only conserves energy but also ensures sustained learning and inference capabilities under power constraints. Our preliminary results demonstrate that this strategy provides 6 to 22 percent accuracy improvements compared to the state of the art with less than 5 percent additional compute. This paper details the development of the device model, describes the integration of energy profiles with intermittency aware dropout and quantization algorithms, and presents a comprehensive evaluation of the proposed approach using real-world energy harvesting data.