MDP-based Energy-aware Task Scheduling for Battery-less IoT

TL;DR

This paper introduces an MDP-based scheduling framework for battery-less IoT devices that optimizes energy use and task completion under intermittent energy harvesting conditions.

Contribution

It formulates the scheduling problem as a Markov decision process and derives an optimal threshold-based policy for reliable task execution.

Findings

OSTB scheduler outperforms baselines in completion rate and latency.

The MDP is unichain with a threshold structure.

Framework applies to both i.i.d. and correlated energy harvesting models.

Abstract

Battery-less Internet of Things (IoT) devices rely on ambient energy harvesting and therefore require scheduling policies that jointly account for energy intermittency and hard timing constraints. This challenge is especially acute in periodic monitoring applications, where a sensing--computing--transmitting task chain must be completed within each reporting cycle. In this paper, we formulate this problem within a setting characterized by independently and identically distributed (i.i.d.) energy arrivals as a long-term average-reward Markov decision process (MDP) that explicitly captures capacitor-voltage evolution, task ordering, permissible start windows, and safe-execution requirements. We further propose rewards that promote reliable task completion while penalizing risky low-energy execution. We prove that the considered MDP is unichain and that the optimal stationary policy has a threshold structure, which leads to an optimal stationary threshold-based (OSTB) scheduler. To account for more realistic energy sources, we additionally study a correlated harvesting model based on a finite-state Markov process and show that the proposed framework can be applied to this richer setting under conservative sufficient conditions. Finally, numerical results show that OSTB outperforms representative baselines in terms of long-term full-chain completion rate, power failures, and latency, particularly when harvested energy is scarce.