Lexicographic Multi-Objective Stochastic Shortest Path with Mixed Max-Sum Costs

TL;DR

This paper introduces a novel approach for stochastic shortest path problems with max-aggregation costs, addressing safety-critical constraints by combining max and sum objectives through a lexicographic value iteration method.

Contribution

It develops a new framework for mixed max-sum cost objectives in SSPs, including an augmented MDP, a finite-horizon solution for cyclic policies, and a lexicographic algorithm for LTL specifications.

Findings

Effective handling of bottleneck and cumulative costs in SSPs.

Successful application to gridworld case studies.

Resolution of zero-marginal-cost cycle issues.

Abstract

We study the Stochastic Shortest Path (SSP) problem for autonomous systems with mixed max-sum cost aggregations under Linear Temporal Logic constraints. Classical SSP formulations rely on sum-aggregated costs, which are suitable for cumulative quantities such as time or energy but fail to capture bottleneck-style objectives such as avoiding high-risk transitions, where performance is determined by the worst single event along a trajectory. Such objectives are particularly important in safety-critical systems, where even one hazardous transition can be unacceptable. To address this limitation, we introduce max-aggregated objectives that minimize the bottleneck cost, i.e., the maximum one-step cost along a trajectory. We show that standard Bellman equations on the original state space do not apply in this setting and propose an augmented MDP with a state variable tracking the running maximum cost, together with a value iteration algorithm. We further identify a cyclic policy phenomenon, where zero-marginal-cost cycles prevent goal reaching under max-aggregation, and resolve it via a finite-horizon formulation. To handle richer task requirements, linear temporal logic specifications are translated into deterministic finite automata and combined with the system to construct a product MDP. We propose a lexicographic value iteration algorithm that handles mixed max-sum objectives under lexicographic ordering on this product MDP. Gridworld case studies demonstrate the effectiveness of the proposed framework.