Semantic-aware Sampling and Transmission in Energy Harvesting Systems: A POMDP Approach

TL;DR

This paper develops a POMDP-based framework for optimizing sampling and transmission in energy harvesting systems, considering semantic-aware metrics like distortion and AoII, and introduces reinforcement learning solutions for complex scenarios.

Contribution

It formulates a novel POMDP approach for joint sampling and transmission optimization with semantic metrics, including a deep RL policy for AoII in general settings.

Findings

Effective belief space truncation for finite-state MDPs.

Revealed non-monotonic switching structure of optimal policies.

Demonstrated policy effectiveness through simulations.

Abstract

We address the problem of real-time remote tracking of a partially observable Markov source in an energy harvesting system with an unreliable communication channel. We consider both sampling and transmission costs. Different from most prior studies that assume the source is fully observable, the sampling cost renders the source partially observable. The goal is to jointly optimize sampling and transmission policies for two semantic-aware metrics: i) a general distortion measure and ii) the age of incorrect information (AoII). We formulate a stochastic control problem. To solve the problem for each metric, we cast a partially observable Markov decision process (POMDP), which is transformed into a belief MDP. Then, for both AoII under the perfect channel setup and distortion, we express the belief as a function of the age of information (AoI). This expression enables us to effectively truncate the corresponding belief space and formulate a finite-state MDP problem, which is solved using the relative value iteration algorithm. For the AoII metric in the general setup, a deep reinforcement learning policy is proposed to solve the belief MDP problem. Simulation results show the effectiveness of the derived policies and, in particular, reveal a non-monotonic switching-type structure of the real-time optimal policy with respect to AoI.