Multi-agent Learning for Cooperative Large-scale Caching Networks

TL;DR

This paper introduces CoM-Cache, a multi-agent reinforcement learning approach for dynamic, scalable cache placement in large-scale networks, improving hit ratios by adapting to content popularity changes.

Contribution

It presents a flexible, scalable multi-agent RL framework for cache placement that effectively handles network dynamics and interactions.

Findings

CoM-Cache outperforms baseline caching schemes in experiments.

The approach adapts to content popularity dynamics in real-time.

It manages complexity through exploiting locality among caches.

Abstract

Caching networks are designed to reduce traffic load at backhaul links, by serving demands from edge-nodes. In the past decades, many studies have been done to address the caching problem. However, in practice, finding an optimal caching policy is still challenging due to dynamicity of traffic and scalability caused by complex impact of caching strategy chosen by each individual cache on other parts of network. In this paper, we focus on cache placement to optimize the performance metrics such as hit ratio in cooperative large-scale caching networks. Our proposed solution, cooperative multi-agent based cache placement (CoM-Cache) is based on multi-agent reinforcement learning framework and can seamlessly track the content popularity dynamics in an on-line fashion. CoM-Cache is enable to solve the problems over a spectrum from isolated to interconnected caches and is designed flexibly to fit any caching networks. To deal with dimensionality issue, CoM-Cache exploits the property of locality of interactions among caches. The experimental results report CoM-Cache outperforms base-line schemes, however at the expense of reasonable additional complexity.