AoI-MDP: An AoI Optimized Markov Decision Process (Student Abstract)

TL;DR

This paper introduces AoI-MDP, a reinforcement learning framework that models observation delays to optimize information freshness in underwater autonomous vehicle tasks.

Contribution

It presents a novel AoI-MDP model that incorporates delay modeling and wait times, improving decision-making for underwater exploration.

Findings

AoI-MDP outperforms standard MDP in simulations.

The approach enhances information freshness and decision accuracy.

Code is available at https://github.com/Xiboxtg/AoI-MDP.

Abstract

Ocean exploration places high demands on autonomous underwater vehicles, especially when there's observation delay. We propose age of information optimized Markov decision process (AoI-MDP) to enhance underwater tasks by modeling observation delay as signal delay and including it in the state space. AoI-MDP also introduces wait time in the action space and integrates AoI with reward functions, optimizing information freshness and decision-making using reinforcement learning. Simulations show AoI-MDP outperforms the standard MDP, demonstrating superior performance, feasibility, and generalization in underwater tasks. To accelerate relevant research, we have made the codes available as open-source at https://github.com/Xiboxtg/AoI-MDP.