A Novel MDP Decomposition Framework for Scalable UAV Mission Planning in Complex and Uncertain Environments

TL;DR

This paper introduces a scalable, fault-tolerant framework for UAV mission planning that decomposes large MDPs into smaller subproblems, enabling real-time decision-making in complex, uncertain environments with provable policy optimality.

Contribution

It proposes a novel two-stage MDP decomposition strategy with theoretical guarantees, significantly improving computational efficiency for UAV mission management.

Findings

Orders-of-magnitude reduction in computation time

Maintains mission reliability and policy optimality

Validated through extensive simulations

Abstract

This paper presents a scalable and fault-tolerant framework for unmanned aerial vehicle (UAV) mission management in complex and uncertain environments. The proposed approach addresses the computational bottleneck inherent in solving large-scale Markov Decision Processes (MDPs) by introducing a two-stage decomposition strategy. In the first stage, a factor-based algorithm partitions the global MDP into smaller, goal-specific sub-MDPs by leveraging domain-specific features such as goal priority, fault states, spatial layout, and energy constraints. In the second stage, a priority-based recombination algorithm solves each sub-MDP independently and integrates the results into a unified global policy using a meta-policy for conflict resolution. Importantly, we present a theoretical analysis showing that, under mild probabilistic independence assumptions, the combined policy is provably equivalent to the optimal global MDP policy. Our work advances artificial intelligence (AI) decision scalability by decomposing large MDPs into tractable subproblems with provable global equivalence. The proposed decomposition framework enhances the scalability of Markov Decision Processes, a cornerstone of sequential decision-making in artificial intelligence, enabling real-time policy updates for complex mission environments. Extensive simulations validate the effectiveness of our method, demonstrating orders-of-magnitude reduction in computation time without sacrificing mission reliability or policy optimality. The proposed framework establishes a practical and robust foundation for scalable decision-making in real-time UAV mission execution.