From Pixels to CSI: Distilling Latent Dynamics For Efficient Wireless Resource Management

TL;DR

This paper introduces a novel machine learning framework that jointly models control system dynamics and wireless channel states in latent space, enabling efficient resource management with significant power savings while maintaining control performance.

Contribution

It proposes a coupled joint-embedding predictive architecture and deep reinforcement learning approach to optimize wireless resource management based on latent control and channel dynamics.

Findings

Reduces transmit power by over 50%.

Maintains control performance comparable to non-optimized methods.

Validates approach on synthetic multimodal data.

Abstract

In this work, we aim to optimize the radio resource management of a communication system between a remote controller and its device, whose state is represented through image frames, without compromising the performance of the control task. We propose a novel machine learning (ML) technique to jointly model and predict the dynamics of the control system as well as the wireless propagation environment in latent space. Our method leverages two coupled joint-embedding predictive architectures (JEPAs): a control JEPA models the control dynamics and guides the predictions of a wireless JEPA, which captures the dynamics of the device's channel state information (CSI) through cross-modal conditioning. We then train a deep reinforcement learning (RL) algorithm to derive a control policy from latent control dynamics and a power predictor to estimate scheduling intervals with favorable channel conditions based on latent CSI representations. As such, the controller minimizes the usage of radio resources by utilizing the coupled JEPA networks to imagine the device's trajectory in latent space. We present simulation results on synthetic multimodal data and show that our proposed approach reduces transmit power by over 50% while maintaining control performance comparable to baseline methods that do not account for wireless optimization.