Cooperative Learning-Based Framework for VNF Caching and Placement Optimization over Low Earth Orbit Satellite Networks

TL;DR

This paper proposes a cooperative learning framework combining Multi-Agent Q-Learning and Bayesian Optimization to optimize VNF caching and placement in Low Earth Orbit Satellite Networks, reducing delay and improving service rates.

Contribution

It introduces an online MAQL method for VNF placement and a BO-based mechanism for caching, addressing complexity and convergence issues in satellite network optimization.

Findings

MAQL achieves near-optimal delay performance similar to dynamic programming.

BO-based caching improves request serving rate over time.

The framework outperforms existing baseline methods.

Abstract

Low Earth Orbit Satellite Networks (LSNs) are integral to supporting a broad range of modern applications, which are typically modeled as Service Function Chains (SFCs). Each SFC is composed of Virtual Network Functions (VNFs), where each VNF performs a specific task. In this work, we tackle two key challenges in deploying SFCs across an LSN. Firstly, we aim to optimize the long-term system performance by minimizing the average end-to-end SFC execution delay, given that each satellite comes with a pre-installed/cached subset of VNFs. To achieve optimal SFC placement, we formulate an offline Dynamic Programming (DP) equation. To overcome the challenges associated with DP, such as its complexity, the need for probability knowledge, and centralized decision-making, we put forth an online Multi-Agent Q-Learning (MAQL) solution. Our MAQL approach addresses convergence issues in the non-stationary LSN environment by enabling satellites to share learning parameters and update their Q-tables based on distinct rules for their selected actions. Secondly, to determine the optimal VNF subsets for satellite caching, we develop a Bayesian Optimization (BO)-based learning mechanism that operates both offline and continuously in the background during runtime. Extensive experiments demonstrate that our MAQL approach achieves near-optimal performance comparable to the DP model and significantly outperforms existing baselines. Moreover, the BO-based approach effectively enhances the request serving rate over time.