Constrained MARL for Coexisting TN-NTN Resource Allocation: Scalability and Flexibility

TL;DR

This paper introduces a scalable and flexible constrained multi-agent reinforcement learning approach for joint terrestrial and non-terrestrial network resource allocation, effectively handling large-scale, dynamic environments.

Contribution

It proposes a decomposition-based learning solution with a stochastic training environment to improve scalability and adaptability in complex TN-NTN resource management.

Findings

Significant scalability improvements over existing methods.

Robust performance in highly dynamic scenarios.

Effective handling of large numbers of channels and users.

Abstract

This paper considers the joint TN-NTN constrained resource allocation, where terrestrial base stations and non-terrestrial base stations coexist in the spectrum. We focus on large-scale and practical scenarios characterized by large numbers of transmission channels and users, alongside highly dynamic user behaviors. As common learning solutions fail to address these challenges, we propose a decomposition solution based on the special properties of the cross-segment interference, and then tackle the original problem via solving subproblems in a sequential learning manner. Furthermore, to enhance the flexibility of the learned policies, we design a stochastic training environment that captures the key characteristics of real-world systems. Simulation results tested on the full 20MHz bandwidth with various numerologies show that our solution significantly improves scalability compared to existing solutions and remains robust in highly dynamic scenarios.