Learning-based Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation

TL;DR

This paper introduces a two-tiered online optimization framework combining reinforcement learning and MILP to improve resource allocation in large-scale data centers, achieving better performance and speed.

Contribution

It presents a novel two-tiered approach integrating RL and MILP for dynamic resource management in data centers, with interpretability and efficiency improvements.

Findings

Outperforms pure MIP solver by over 15%.

Achieves 100x speedup in computation.

Effectively handles large-scale RAS problems.

Abstract

Online optimization of resource management for large-scale data centers and infrastructures to meet dynamic capacity reservation demands and various practical constraints (e.g., feasibility and robustness) is a very challenging problem. Mixed Integer Programming (MIP) approaches suffer from recognized limitations in such a dynamic environment, while learning-based approaches may face with prohibitively large state/action spaces. To this end, this paper presents a novel two-tiered online optimization to enable a learning-based Resource Allowance System (RAS). To solve optimal server-to-reservation assignment in RAS in an online fashion, the proposed solution leverages a reinforcement learning (RL) agent to make high-level decisions, e.g., how much resource to select from the Main Switch Boards (MSBs), and then a low-level Mixed Integer Linear Programming (MILP) solver to generate the local server-to-reservation mapping, conditioned on the RL decisions. We take into account fault tolerance, server movement minimization, and network affinity requirements and apply the proposed solution to large-scale RAS problems. To provide interpretability, we further train a decision tree model to explain the learned policies and to prune unreasonable corner cases at the low-level MILP solver, resulting in further performance improvement. Extensive evaluations show that our two-tiered solution outperforms baselines such as pure MIP solver by over $15\%$ while delivering $100\times$ speedup in computation.