Sampling to Achieve the Goal: An Age-aware Remote Markov Decision Process

TL;DR

This paper explores how Age of Information (AoI) impacts remote decision-making, proposing an age-aware MDP framework that links AoI directly to decision utility rather than just minimizing AoI.

Contribution

It introduces an age-aware remote MDP model that connects AoI with decision utility, moving beyond traditional AoI minimization approaches.

Findings

Age-aware remote MDP reduces to standard MDP without delays.

AoI serves as side information, not just an optimization metric.

Directly optimizing decision utility improves remote decision-making.

Abstract

Age of Information (AoI) has been recognized as an important metric to measure the freshness of information. Central to this consensus is that minimizing AoI can enhance the freshness of information, thereby facilitating the accuracy of subsequent decision-making processes. However, to date the direct causal relationship that links AoI to the utility of the decision-making process is unexplored. To fill this gap, this paper provides a sampling-control co-design problem, referred to as an age-aware remote Markov Decision Process (MDP) problem, to explore this unexplored relationship. Our framework revisits the sampling problem in [1] with a refined focus: moving from AoI penalty minimization to directly optimizing goal-oriented remote decision-making process under random delay. We derive that the age-aware remote MDP problem can be reduced to a standard MDP problem without delays, and reveal that treating AoI solely as a metric for optimization is not optimal in achieving remote decision making. Instead, AoI can serve as important side information to facilitate remote decision making.