Approximate dynamic programming using fluid and diffusion approximations with applications to power management

TL;DR

This paper introduces a novel approach to approximate dynamic programming by leveraging fluid and diffusion approximations, demonstrated through a power management application in computer processors.

Contribution

It proposes using fluid and diffusion approximations to guide the selection of function classes in neuro-dynamic programming, enhancing approximation accuracy.

Findings

Effective power management strategies derived from the approach.

Improved approximation of dynamic programming solutions.

Application to processor speed scaling demonstrates practical benefits.

Abstract

Neuro-dynamic programming is a class of powerful techniques for approximating the solution to dynamic programming equations. In their most computationally attractive formulations, these techniques provide the approximate solution only within a prescribed finite-dimensional function class. Thus, the question that always arises is how should the function class be chosen? The goal of this paper is to propose an approach using the solutions to associated fluid and diffusion approximations. In order to illustrate this approach, the paper focuses on an application to dynamic speed scaling for power management in computer processors.