Optimal Control Problem in a Stochastic Model with Periodic Hits on the Boundary of a Given Subset of the State Set (Tuning Problem)

TL;DR

This paper introduces a stochastic control model with controls applied at boundary hits, providing new representations of control quality and demonstrating that optimal controls are deterministic, with potential applications in technical and economic systems.

Contribution

The paper formulates a general stochastic control model with boundary-based controls, deriving explicit extremal conditions and demonstrating the determinism of optimal control strategies.

Findings

Control strategies are deterministic and determined by extremum points.

New linear-fractional integral functional representations of the stationary cost index.

Explicit analytic forms for optimal control functions are obtained.

Abstract

In this paper, a general stochastic model with controls applied at the moments when the random process hits the boundary of a given subset of the state set is proposed and studied. The general concept of the model is formulated and its possible applications in technical and economic systems are described. Two versions of the general stochastic model, the version based on the use of a continuous-time semi-Markov process with embedded absorbing Markov chain and the version based on the use of a discrete-time Markov process with absorbing states, are analyzed. New representations of the stationary cost index of the control quality are obtained for both versions. It is shown that this index can be represented as a linear-fractional integral functional of two discrete probability distributions determining the control strategy. The results obtained by the author of this paper about an extremum of such functionals were used to prove that, in both versions of the model, the control is deterministic and is determined by the extremum points of functions of two discrete arguments for which the explicit analytic representations are obtained. The perspectives of the further development of this stochastic model are outlined.