Reinforcement Learning for Opportunistic Routing in Software-Defined LEO-Terrestrial Systems

TL;DR

This paper introduces a reinforcement learning-based opportunistic routing strategy for LEO satellite networks, leveraging SDN control to reduce data delivery delay amidst rapidly changing topologies.

Contribution

It formulates a stochastic optimization problem and applies residual reinforcement learning to optimize routing, a novel approach for dynamic LEO-terrestrial systems.

Findings

Significant reduction in queue lengths compared to classical algorithms

Improved delay performance in simulated orbital scenarios

Effective adaptation to highly dynamic network topologies

Abstract

The proliferation of large-scale low Earth orbit (LEO) satellite constellations is driving the need for intelligent routing strategies that can effectively deliver data to terrestrial networks under rapidly time-varying topologies and intermittent gateway visibility. Leveraging the global control capabilities of a geostationary (GEO)-resident software-defined networking (SDN) controller, we introduce opportunistic routing, which aims to minimize delivery delay by forwarding packets to any currently available ground gateways rather than fixed destinations. This makes it a promising approach for achieving low-latency and robust data delivery in highly dynamic LEO networks. Specifically, we formulate a constrained stochastic optimization problem and employ a residual reinforcement learning framework to optimize opportunistic routing for reducing transmission delay. Simulation results over multiple days of orbital data demonstrate that our method achieves significant improvements in queue length reduction compared to classical backpressure and other well-known queueing algorithms.