AoI and Energy Consumption Oriented Dynamic Status Updating in Caching Enabled IoT Networks

TL;DR

This paper proposes a reinforcement learning-based strategy for dynamic status updates in caching-enabled IoT networks, balancing information freshness and energy consumption to optimize sensor performance.

Contribution

It introduces a novel MDP formulation and a model-free reinforcement learning algorithm to optimize status updates considering AoI and energy use.

Findings

The proposed algorithm converges effectively in simulations.

It outperforms the zero-wait baseline policy.

The approach balances AoI and energy consumption efficiently.

Abstract

Caching has been regarded as a promising technique to alleviate energy consumption of sensors in Internet of Things (IoT) networks by responding to users' requests with the data packets stored in the edge caching node (ECN). For real-time applications in caching enabled IoT networks, it is essential to develop dynamic status update strategies to strike a balance between the information freshness experienced by users and energy consumed by the sensor, which, however, is not well addressed. In this paper, we first depict the evolution of information freshness, in terms of age of information (AoI), at each user. Then, we formulate a dynamic status update optimization problem to minimize the expectation of a long term accumulative cost, which jointly considers the users' AoI and sensor's energy consumption. To solve this problem, a Markov Decision Process (MDP) is formulated to cast the status updating procedure, and a model-free reinforcement learning algorithm is proposed, with which the challenge brought by the unknown of the formulated MDP's dynamics can be addressed. Finally, simulations are conducted to validate the convergence of our proposed algorithm and its effectiveness compared with the zero-wait baseline policy.