Latency-Optimal Cache-aided Multicast Streaming via Forward-Backward Reinforcement Learning

TL;DR

This paper introduces a forward-backward reinforcement learning approach to optimize latency and other metrics in cache-enabled multicast streaming networks, effectively capturing network dynamics for improved performance.

Contribution

It develops a novel forward-backward Markov decision process and a corresponding reinforcement learning algorithm for latency optimization in cache-aided multicast streaming.

Findings

The FB-MORL algorithm effectively reduces latency.

It improves outage probability and resource efficiency.

Simulation results demonstrate superior performance over baseline methods.

Abstract

We consider a cellular network equipped with cache-enabled base-stations (BSs) leveraging an orthogonal multipoint multicast (OMPMC) streaming scheme. The network operates in a time-slotted fashion to serve content-requesting users by streaming cached files. The users being unsatisfied by the multicat streaming face a delivery outage, implying that they will remain interested in their preference at the next time-slot, which leads to a forward dynamics on the user preference. To design a latency-optimal streaming policy, the dynamics of latency is properly modeled and included in the learning procedure. We show that this dynamics surprisingly represents a backward dynamics. The combination of problem's forward and backward dynamics then develops a forward-backward Markov decision process (FB-MDP) that fully captures the network evolution across time. This FB-MDP necessitates usage of a forward-backward multi-objective reinforcement learning (FB-MORL) algorithm to optimize the expected latency as well as other performance metrics of interest including the overall outage probability and total resource consumption. Simulation results show the merit of proposed FB-MORL algorithm in finding a promising dynamic cache policy.