A Persistence-Aware Framework for Age Violation Control in Wireless Status Update Systems

TL;DR

This paper introduces a persistence-aware reliability framework for wireless status updates, utilizing a novel C-AVR metric and a distributional reinforcement learning approach to better capture consecutive age violations.

Contribution

It develops a new C-AVR metric for persistence-aware reliability and proposes a QR-D3QN-based method to optimize it, addressing temporal correlations in violations.

Findings

QR-D3QN outperforms expectation-based baselines in simulations.

Distributional learning improves reliability for long violation sequences.

The framework effectively captures tail-sensitive persistence objectives.

Abstract

Timely and reliable status updates are essential for emerging QoS-sensitive wireless applications. Common age of information (AoI)-based metrics, such as average AoI and age violation rate (AVR), characterize time-averaged freshness or violation frequency but do not explicitly capture the temporal persistence of consecutive age violations, which can be critical in safety-sensitive wireless applications. We develop a persistence-aware reliability framework based on the consecutive age violation rate (C-AVR) vector, whose components quantify AoI threshold violations over consecutive time windows of different lengths. Through flexible weighting schemes, the proposed framework unifies reliability objectives ranging from average persistence to tail-sensitive performance. Optimizing weighted C-AVR objectives is challenging because consecutive violations are temporally correlated, leading to sparse learning signals. To address this issue, we develop a distributional reinforcement learning approach based on a quantile regression dueling double deep Q-network (QR-D3QN). By modeling a quantile-based return distribution rather than only a scalar expected return, QR-D3QN provides richer value-estimation signals for rare but prolonged violation sequences under stochastic packet arrivals, unreliable channels, and transmission cost constraints. Simulation results show that QR-D3QN consistently outperforms expectation-based baselines across a wide range of weighting schemes and system settings, with particularly significant gains under tail-sensitive persistence objectives. Component-wise analysis further shows that distributional value learning substantially improves reliability across multiple persistence scales, especially for long consecutive violation sequences. Overall, our results establish the proposed C-AVR framework as an effective foundation for persistence-aware reliability evaluation.