On Sequential Fault-Intolerant Process Planning

TL;DR

This paper introduces the Sequential Fault-Intolerant Process Planning problem, proposing tight online algorithms for deterministic and probabilistic settings, with empirical results showing the advantage of specialized algorithms over general ones.

Contribution

It formulates SFIPP, a new planning problem with non-additive rewards, and develops provably optimal algorithms for both deterministic and probabilistic scenarios.

Findings

Specialized algorithms outperform general algorithms in empirical tests.

Effective balancing of exploration and exploitation in probabilistic SFIPP.

Provably tight algorithms for deterministic SFIPP.

Abstract

We propose and study a planning problem we call Sequential Fault-Intolerant Process Planning (SFIPP). SFIPP captures a reward structure common in many sequential multi-stage decision problems where the planning is deemed successful only if all stages succeed. Such reward structures are different from classic additive reward structures and arise in important applications such as drug/material discovery, security, and quality-critical product design. We design provably tight online algorithms for settings in which we need to pick between different actions with unknown success chances at each stage. We do so both for the foundational case in which the behavior of actions is deterministic, and the case of probabilistic action outcomes, where we effectively balance exploration for learning and exploitation for planning through the usage of multi-armed bandit algorithms. In our empirical evaluations, we demonstrate that the specialized algorithms we develop, which leverage additional information about the structure of the SFIPP instance, outperform our more general algorithm.