Tracking and Assigning Jobs to a Markov Machine
Subhankar Banerjee, Sennur Ulukus

TL;DR
This paper models a communication system with a machine and server, optimizing sampling and job assignment policies to minimize costs related to machine state accuracy and job dropping, using Markov decision processes.
Contribution
It introduces a Markov decision process framework for optimal sampling and job assignment in a system with state estimation and penalties, revealing threshold policies as optimal.
Findings
Optimal threshold policy identified for the system.
Necessary and sufficient conditions for policy optimality derived.
Optimal threshold computed without state space bounds.
Abstract
We consider a time-slotted communication system with a machine, a cloud server, and a sampler. Job requests from the users are queued on the server to be completed by the machine. The machine has two states, namely, a busy state and a free state. The server can assign a job to the machine in a first-in-first-served manner. If the machine is free, it completes the job request from the server; otherwise, it drops the request. Upon dropping a job request, the server is penalized. When the machine is in the free state, the machine can get into the busy state with an internal job. When the server does not assign a job request to the machine, the state of the machine evolves as a symmetric Markov chain. If the machine successfully accepts the job request from the server, the state of the machine goes to the busy state and follows a different dynamics compared to the dynamics when the machine…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsScheduling and Optimization Algorithms
