First Passage Value
Cenk Oguz Saglam, Katie Byl

TL;DR
This paper extends the concept of Mean First Passage Time to a more general framework called Mean First Passage Value, providing bounds and demonstrating its effectiveness in analyzing complex, metastable stochastic systems, especially Markov Decision Processes.
Contribution
It introduces the Mean First Passage Value (MFPV), generalizes bounds for First Passage Value, and shows that system-wide MFPT can be captured using the second largest eigenvalue in complex systems.
Findings
System-wide MFPT can be approximated using the second largest eigenvalue.
MFPV can represent various quantities like energy, distance, or time.
Bounds on FPV are provided for specified confidence levels.
Abstract
For many stochastic dynamic systems, the Mean First Passage Time (MFPT) is a useful concept, which gives expected time before a state of interest. This work is an extension of MFPT in several ways. (1) We show that for some systems the system-wide MFPT, calculated using the second largest eigenvalue only, captures most of the fundamental dynamics, even for quite complex, high-dimensional systems. (2) We generalize MFPT to Mean First Passage Value (MFPV), which gives a more general value of interest, e.g., energy expenditure, distance, or time. (3) We provide bounds on First Passage Value (FPV) for a given confidence level. At the heart of this work, we emphasize that for our goals, many hybrid systems can be approximated as Markov Decision Processes. So, many systems can be controlled effectively using this framework. However, our framework is particularly useful for metastable systems.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Thermodynamics and Statistical Mechanics · stochastic dynamics and bifurcation · Probabilistic and Robust Engineering Design
