Status Updating in Two-Way Delay Systems with Preemption

TL;DR

This paper analyzes a two-way status update system with preemption, modeling delays as geometric servers, and derives an optimal policy to minimize the age of information using Markov decision processes.

Contribution

It introduces a novel model for two-way status updates with preemption and derives the optimal request generation policy via MDP and RVI algorithms.

Findings

Optimal policy has a threshold structure.

System outperforms baseline policies.

Convergence of the RVI algorithm is proven.

Abstract

We consider a status update system consisting of a sampler, a sink, and a controller located at the sink. The controller sends requests to the sampler to generate and transmit status updates. Packet transmissions from the controller to the sampler (reverse link) and from the sampler to the sink (forward link) experience random delays. The reverse and forward links are modeled as servers with geometric service times, referred to as the controller and sampler servers, respectively. Each server is equipped with a single buffer that stores an arriving packet when the server is busy. We adopt a preemption-in-waiting policy on both links, whereby an arriving packet replaces the packet in the buffer whenever the buffer is full. Our main goal is to determine the optimal generation times of request packets at the controller in order to minimize the long-term average age of information (AoI) at the sink. We formulate the problem as a Markov decision process (MDP) and derive the optimal stationary deterministic policy using the relative value iteration (RVI) algorithm. We prove the convergence of the algorithm. Numerical results show that the proposed system consistently outperforms baseline policies from prior work and reveal a threshold-based structure for the optimal policy.