Nash Soft Actor-Critic LEO Satellite Handover Management Algorithm for Flying Vehicles

TL;DR

This paper introduces Nash-SAC, a multi-agent reinforcement learning-based strategy for LEO satellite handover management, significantly reducing handovers and blocking rates while improving network utility for flying vehicles and ground users.

Contribution

The paper proposes a novel distributed handover strategy using MARL and game theory, outperforming traditional methods in reducing handovers and enhancing network utility.

Findings

Reduces handovers by over 16%

Decreases blocking rate by over 18%

Improves network utility by up to 48%

Abstract

Compared with the terrestrial networks (TN), which can only support limited coverage areas, low-earth orbit (LEO) satellites can provide seamless global coverage and high survivability in case of emergencies. Nevertheless, the swift movement of the LEO satellites poses a challenge: frequent handovers are inevitable, compromising the quality of service (QoS) of users and leading to discontinuous connectivity. Moreover, considering LEO satellite connectivity for different flying vehicles (FVs) when coexisting with ground terminals, an efficient satellite handover decision control and mobility management strategy is required to reduce the number of handovers and allocate resources that align with different users' requirements. In this paper, a novel distributed satellite handover strategy based on Multi-Agent Reinforcement Learning (MARL) and game theory named Nash-SAC has been proposed to solve these problems. From the simulation results, the Nash-SAC-based handover strategy can effectively reduce the handovers by over 16 percent and the blocking rate by over 18 percent, outperforming local benchmarks such as traditional Q-learning. It also greatly improves the network utility used to quantify the performance of the whole system by up to 48 percent and caters to different users requirements, providing reliable and robust connectivity for both FVs and ground terminals.