Age of Estimates: When to Submit Jobs to a Markov Machine to Maximize Revenue

TL;DR

This paper develops strategies for optimally submitting jobs to Markov machines to maximize revenue, using age-based policies that adapt to the machine's state and dynamics.

Contribution

It introduces a novel approach to maximize MM utility by deriving optimal submission policies based on age of estimate, considering Poisson arrivals and sampling.

Findings

Optimal policies are threshold or switching based on age of estimate.

The approach improves revenue maximization over naive strategies.

Policy derivation depends on Markov machine parameters.

Abstract

With the dawn of AI factories ushering a new era of computing supremacy, development of strategies to effectively track and utilize the available computing resources is garnering utmost importance. These computing resources are often modeled as Markov sources, which oscillate between free and busy states, depending on their internal load and external utilization, and are commonly referred to as Markov machines (MMs). Most of the prior work solely focuses on the problem of tracking these MMs, while often assuming a rudimentary decision process that governs their utilization. Our key observation is that the ultimate goal of tracking a MM is to properly utilize it. In this work, we consider the problem of maximizing the utility of a MM, where the utility is defined as the average revenue generated by the MM. Assuming a Poisson job arrival process and a query-based sampling procedure to sample the state of the MM, we find the optimal times to submit the available jobs to the MM so as to maximize the average revenue generated per unit job. We show that, depending on the parameters of the MM, the optimal policy is in the form of either a \emph{threshold policy} or a \emph{switching policy} based on the \emph{age of our estimate} of the state of the MM.