Latency-aware Human-in-the-Loop Reinforcement Learning for Semantic Communications

TL;DR

This paper presents a latency-aware reinforcement learning framework that incorporates human feedback for semantic communication, ensuring timely task execution while maintaining semantic fidelity in RAN systems.

Contribution

It introduces a novel constrained RL approach with human-in-the-loop feedback and latency control for semantic communication in wireless networks.

Findings

Successfully meets per-user timing constraints in simulations

Outperforms baseline schedulers in reward metrics

Stabilizes resource consumption in heterogeneous deadline scenarios

Abstract

Semantic communication promises task-aligned transmission but must reconcile semantic fidelity with stringent latency guarantees in immersive and safety-critical services. This paper introduces a time-constrained human-in-the-loop reinforcement learning (TC-HITL-RL) framework that embeds human feedback, semantic utility, and latency control within a semantic-aware Open radio access network (RAN) architecture. We formulate semantic adaptation driven by human feedback as a constrained Markov decision process (CMDP) whose state captures semantic quality, human preferences, queue slack, and channel dynamics, and solve it via a primal--dual proximal policy optimization algorithm with action shielding and latency-aware reward shaping. The resulting policy preserves PPO-level semantic rewards while tightening the variability of both air-interface and near-real-time RAN intelligent controller processing budgets. Simulations over point-to-multipoint links with heterogeneous deadlines show that TC-HITL-RL consistently meets per-user timing constraints, outperforms baseline schedulers in reward, and stabilizes resource consumption, providing a practical blueprint for latency-aware semantic adaptation.