# Optimal Asynchronous Dynamic Policies in Energy-Efficient Data Centers

## TL;DR

This paper develops an optimal asynchronous control policy for energy-efficient data centers with heterogeneous servers, using Markov decision processes to maximize long-term profit and demonstrate the optimality of bang-bang control.

## Contribution

It introduces a novel Markov decision process framework for asynchronous policies in data centers, proving bang-bang control as always optimal and characterizing threshold-based policies.

## Key findings

- Bang-bang control is proven to be optimal.
- Threshold-type dynamic control is supported.
- Monotonicity and optimality of long-run profit are characterized.

## Abstract

In this paper, we use a Markov decision process to find optimal asynchronous policy of an energy-efficient data center with two groups of heterogeneous servers, a finite buffer, and a fast setup process at sleep state. Servers in Group 1 always work. Servers in Group 2 may either work or sleep, and a fast setup process occurs when server's states are changed from sleep to work. In such a data center, an asynchronous dynamic policy is designed as two sub-policies: The setup policy and the sleep policy, which determine the switch rule between the work and sleep states for the servers in Group 2. To analyze the optimal asynchronous dynamic policy, we apply the Markov decision process to establish a policy-based Poisson equation, which provides expression for the unique solution of the performance potential by means of the RG-factorization. Based on this, we can characterize the monotonicity and optimality of the long-run average profit of the data center with respect to the asynchronous dynamic policy under different service prices. Furthermore, we prove that the bang-bang control is always optimal for this optimization problem, and supports a threshold-type dynamic control in the energy-efficient data center. We hope that the methodology and results derived in this paper can shed light to the study of more general energy-efficient data centers.

---
Source: https://tomesphere.com/paper/1901.03371