Optimizing Information Freshness in IoT Systems with Update Rate Constraints: A Token-Based Approach

TL;DR

This paper introduces a token-based method to optimize information freshness in IoT systems with update rate constraints, transforming complex CMDPs into simpler MDPs for improved policy performance.

Contribution

The paper presents a novel token-based approach that simplifies solving constrained MDPs in IoT status update systems, enabling efficient optimization of information freshness metrics.

Findings

Token-based approach outperforms baseline policies.

Converges to optimal policy with increasing tokens.

Effective for systems with one or two update constraints.

Abstract

In Internet of Things (IoT) status update systems, where information is sampled and subsequently transmitted from a source to a destination node, the imperative necessity lies in maintaining the timeliness of information and updating the system with optimal frequency. Optimizing information freshness in resource-limited status update systems often involves Constrained Markov Decision Process (CMDP) problems with update rate constraints. Solving CMDP problems, especially with multiple constraints, is a challenging task. To address this, we present a token-based approach that transforms CMDP into an unconstrained MDP, simplifying the solution process. We apply this approach to systems with one and two update rate constraints for optimizing Age of Incorrect Information (AoII) and Age of Information (AoI) metrics, respectively, and explore the analytical and numerical aspects. Additionally, we introduce an iterative triangle bisection method for solving the CMDP problems with two constraints, comparing its results with the token-based MDP approach. Our findings show that the token-based approach yields superior performance over baseline policies, converging to the optimal policy as the maximum number of tokens increases.