Learning Ad Hoc Network Dynamics via Graph-Structured World Models

TL;DR

This paper introduces G-RSSM, a graph-structured world model for learning complex ad hoc network dynamics, enabling size-agnostic decision making through offline training and imagined rollouts.

Contribution

The paper presents G-RSSM, a novel graph-based recurrent state space model that captures per node dynamics and supports scalable, offline policy training in wireless networks.

Findings

The learned policy maintains high connectivity across diverse network scenarios.

G-RSSM effectively models complex, coupled network dynamics.

The approach is size-agnostic, working well from 50 to 1000 nodes.

Abstract

Ad hoc wireless networks exhibit complex, innate and coupled dynamics: node mobility, energy depletion and topology change that are difficult to model analytically. Model-free deep reinforcement learning requires sustained online interaction whereas existing model based approaches use flat state representations that lose per node structure. Therefore we propose G-RSSM, a graph structured recurrent state space model that maintains per node latent states with cross node multi head attention to learn the dynamics jointly from offline trajectories. We apply the proposed method to the downstream task clustering where a cluster head selection policy trains entirely through imagined rollouts in the learned world model. Across 27 evaluation scenarios spanning MANET, VANET, FANET, WSN and tactical networks with N=30 to 1000 nodes, the learned policy maintains high connectivity with only trained for N=50. Herein, we propose the first multi physics graph structured world model applied to combinatorial per node decision making in size agnostic wireless ad hoc networks.