Solving Constrained Stochastic Shortest Path Problems with Scalarisation

TL;DR

This paper introduces CARL, a novel algorithm that efficiently solves constrained stochastic shortest path problems by scalarising costs and solving a series of unconstrained SSPs, outperforming existing methods on benchmarks.

Contribution

The paper presents CARL, a new algorithm that transforms CSSPs into scalarised SSPs, enabling more efficient solutions with heuristic search and subgradient optimization.

Findings

CARL solves 50% more problems than state-of-the-art methods.

It effectively handles probabilistic effects and constraints in shortest path problems.

Experimental results demonstrate significant performance improvements.

Abstract

Constrained Stochastic Shortest Path Problems (CSSPs) model problems with probabilistic effects, where a primary cost is minimised subject to constraints over secondary costs, e.g., minimise time subject to monetary budget. Current heuristic search algorithms for CSSPs solve a sequence of increasingly larger CSSPs as linear programs until an optimal solution for the original CSSP is found. In this paper, we introduce a novel algorithm CARL, which solves a series of unconstrained Stochastic Shortest Path Problems (SSPs) with efficient heuristic search algorithms. These SSP subproblems are constructed with scalarisations that project the CSSP's vector of primary and secondary costs onto a scalar cost. CARL finds a maximising scalarisation using an optimisation algorithm similar to the subgradient method which, together with the solution to its associated SSP, yields a set of policies that are combined into an optimal policy for the CSSP. Our experiments show that CARL solves 50% more problems than the state-of-the-art on existing benchmarks.