Construction of Asymptotically Optimal Control for a Stochastic Network from a Free Boundary Problem

TL;DR

This paper develops an asymptotically optimal control policy for a specific stochastic network by solving a free boundary problem, extending previous diffusion approximation methods to more complex workload processes.

Contribution

It introduces a novel approach using a free boundary problem to derive optimal control policies for a multi-dimensional stochastic network.

Findings

Proposes threshold-based control policies for the network.

Proves the asymptotic optimality of the proposed policies.

Extends diffusion approximation methods to complex workload models.

Abstract

An asymptotic framework for optimal control of multiclass stochastic processing networks, using formal diffusion approximations under suitable temporal and spatial scaling, by Brownian control problems (BCP) and their equivalent workload formulations (EWF), has been developed by Harrison (1988). This framework has been implemented in many works for constructing asymptotically optimal control policies for a broad range of stochastic network models. To date all asymptotic optimality results for such networks correspond to settings where the solution of the EWF is a reflected Brownian motion in the positive orthant with normal reflections. In this work we consider a well studied stochastic network which is perhaps the simplest example of a model with more than one dimensional workload process. In the regime considered here, the singular control problem corresponding to the EWF does not have a simple form explicit solution, however by considering an associated free boundary problem one can give a representation for an optimal controlled process as a two dimensional reflected Brownian motion in a Lipschitz domain whose boundary is determined by the solution of the free boundary problem. Using the form of the optimal solution we propose a sequence of control policies, given in terms of suitable thresholds, for the scaled stochastic network control problems and prove that this sequence of policies is asymptotically optimal. As suggested by the solution of the EWF, the policy we propose requires a server to idle under certain conditions which are specified in terms of the thresholds determined from the free boundary.