Online Learning for Intelligent Thermal Management of Interference-coupled and Passively Cooled Base Stations

TL;DR

This paper proposes an online reinforcement learning approach using SAC to optimize throughput in passive cooled base stations, balancing thermal constraints and interference in dynamic outdoor environments.

Contribution

It introduces a novel RL-based method with a mitigation mechanism for overheating, achieving near-optimal throughput without prior heat dissipation knowledge.

Findings

Achieves up to 88.6% of the global optimum throughput.

Effectively manages overheating risk during RL exploration.

Operates without prior knowledge of heat dissipation efficiency.

Abstract

Passively cooled base stations (PCBSs) have emerged to deliver better cost and energy efficiency. However, passive cooling necessitates intelligent thermal control via traffic management, i.e., the instantaneous data traffic or throughput of a PCBS directly impacts its thermal performance. This is particularly challenging for outdoor deployment of PCBSs because the heat dissipation efficiency is uncertain and fluctuates over time. What is more, the PCBSs are interference-coupled in multi-cell scenarios. Thus, a higher-throughput PCBS leads to higher interference to the other PCBSs, which, in turn, would require more resource consumption to meet their respective throughput targets. In this paper, we address online decision-making for maximizing the total downlink throughput for a multi-PCBS system subject to constraints related on operating temperature. We demonstrate that a reinforcement learning (RL) approach, specifically soft actor-critic (SAC), can successfully perform throughput maximization while keeping the PCBSs cool, by adapting the throughput to time-varying heat dissipation conditions. Furthermore, we design a denial and reward mechanism that effectively mitigates the risk of overheating during the exploration phase of RL. Simulation results show that our approach achieves up to 88.6% of the global optimum. This is very promising, as our approach operates without prior knowledge of future heat dissipation efficiency, which is required by the global optimum.