On viscosity solution of HJB equations with state constraints and reflection control

TL;DR

This paper studies a control problem involving queueing networks with state constraints and reflection controls, establishing the value function as a viscosity solution to a related HJB equation with boundary conditions.

Contribution

It introduces a novel approach to characterize the value function as a viscosity solution for HJB equations with state constraints and reflection controls.

Findings

Value function is a viscosity solution to the HJB equation.

Under certain conditions, the value function is unique.

The framework applies to queueing network control problems.

Abstract

Motivated by a control problem of a certain queueing network we consider a control problem where the dynamics is constrained in the nonnegative orthant $\mathbb{R}_+$ of the $d$-dimensional Euclidean space and controlled by the reflections at the faces/boundaries. We define a discounted value function associated to this problem and show that the value function is a viscosity solution to a certain HJB equation in $\mathbb{R}_+$ with nonlinear Neumann type boundary condition. Under certain conditions, we also characterize this value function as the unique solution to this HJB equation.