# Hybrid Planning for Dynamic Multimodal Stochastic Shortest Paths

## TL;DR

This paper introduces DMSSPs, a new class of MDPs for complex decision-making under uncertainty, and proposes HSP, a hybrid algorithm that improves planning efficiency and solution quality in dynamic multimodal environments.

## Contribution

The paper presents a novel class of MDPs called DMSSPs and a hybrid planning algorithm HSP that combines heuristic search, approximate dynamic programming, and hierarchical planning.

## Key findings

- HSP outperforms state-of-the-art algorithms in autonomous multimodal routing.
- HSP achieves higher quality solutions with efficient computation.
- The approach effectively handles uncertainty and dynamic updates in complex environments.

## Abstract

Sequential decision problems in applications such as manipulation in warehouses, multi-step meal preparation, and routing in autonomous vehicle networks often involve reasoning about uncertainty, planning over discrete modes as well as continuous states, and reacting to dynamic updates. To formalize such problems generally, we introduce a class of Markov Decision Processes (MDPs) called Dynamic Multimodal Stochastic Shortest Paths (DMSSPs). Much of the work in these domains solves deterministic variants, which can yield poor results when the uncertainty has downstream effects. We develop a Hybrid Stochastic Planning (HSP) algorithm, which uses domain-agnostic abstractions to efficiently unify heuristic search for planning over discrete modes, approximate dynamic programming for stochastic planning over continuous states, and hierarchical interleaved planning and execution. In the domain of autonomous multimodal routing, HSP obtains significantly higher quality solutions than a state-of-the-art Upper Confidence Trees algorithm and a two-level Receding Horizon Control algorithm.