Markov Chains with Rewinding
Amir Azarmehr, Soheil Behnezhad, Alma Ghafari, Madhu Sudan

TL;DR
This paper introduces Markov Chains with Rewinding, a new randomized process allowing strategic rewinding to improve state identification efficiency, and analyzes the power and limitations of adaptive versus non-adaptive strategies.
Contribution
It formalizes Markov Chains with Rewinding, proves non-adaptive strategies are as powerful as adaptive ones for distinguishability, and quantifies the efficiency gap with polynomial bounds.
Findings
Non-adaptive strategies can match adaptive strategies in distinguishability.
Adaptive strategies are more query-efficient than non-adaptive ones, with a polynomial gap.
Rewinding strategies enhance the ability to identify states in Markov chains.
Abstract
Motivated by techniques developed in recent progress on lower bounds for sublinear time algorithms (Behnezhad, Roghani and Rubinstein, STOC 2023, FOCS 2023, and STOC 2024) we introduce and study a new class of randomized algorithmic processes that we call Markov Chains with Rewinding. In this setting, an algorithm interacts with a (partially observable) Markovian random evolution by strategically rewinding the Markov chain to previous states. Depending on the application, this may lead the evolution to desired states faster, or allow the agent to efficiently learn or test properties of the underlying Markov chain that may be infeasible or inefficient with passive observation. We study the task of identifying the initial state in a given partially observable Markov chain. Analysis of this question in specific Markov chains is the central ingredient in the above cited works and we aim…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOptimization and Search Problems · Advanced Bandit Algorithms Research · Machine Learning and Algorithms
