A useful technique for piecewise deterministic Markov decision processes

TL;DR

This paper introduces a technique that transforms piecewise deterministic Markov decision processes into auxiliary forms with desirable properties, facilitating analysis and application to risk-sensitive total cost criteria.

Contribution

The paper proposes a novel method to construct auxiliary PDMDPs that preserve performance measures, enabling better analysis of policies with unbounded transition intensities.

Findings

Auxiliary PDMDPs have properties not present in original models.

Performance measures are preserved between original and auxiliary PDMDPs.

Technique applied successfully to risk-sensitive PDMDPs with total cost criteria.

Abstract

This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary PDMDP from the original one. As to be discussed and claified, the auxiliary PDMDP possesses certain desired properties, which may not be possessed by the original PDMDP. Moreover, the performance measure of any policy in the original PDMDP can be replicated by the auxiliary PDMDP for a large class of performance criteria. As an application, we apply this technique to risk-sensitive PDMDPs with total cost criteria.