Optimal Semantic-aware Sampling and Transmission in Energy Harvesting Systems Through the AoII

TL;DR

This paper develops optimal sampling and transmission policies for energy harvesting systems using AoII as a semantic metric, employing POMDP modeling and deep reinforcement learning to minimize information staleness.

Contribution

It introduces a novel approach combining AoII with POMDP and deep RL for energy harvesting status update systems, addressing non-observability and channel imperfections.

Findings

Proposed belief-based MDP solution for perfect channels.

Designed deep RL policy for general scenarios.

Simulation shows improved AoII performance over baselines.

Abstract

We study a real-time tracking problem in an energy harvesting status update system with a Markov source and an imperfect channel, considering both sampling and transmission costs. The problem primary challenge stems from the non-observability of the source due to the sampling cost. By using the age of incorrect information (AoII) as a semantic-aware performance metric, our main goal is to find an optimal policy that minimizes the time average AoII subject to an energy-causality constraint. To this end, a stochastic optimization problem is formulated and solved by modeling it as a partially observable Markov decision process (POMDP). More specifically, to solve the main problem, we use the notion of a belief state and cast the problem as a belief MDP problem. Then, for the perfect channel setup, we effectively truncate the corresponding belief space and solve the MDP problem using the relative value iteration method. For the general setup, a deep reinforcement learning policy is proposed. The simulation results show the efficacy of the derived policies in comparison to an AoI-optimal policy and an opportunistic baseline policy.