Expert-Guided POMDP Learning for Data-Efficient Modeling in Healthcare

TL;DR

This paper presents the Fuzzy MAP EM algorithm, which integrates expert knowledge into POMDP parameter learning, improving data efficiency and robustness in healthcare applications.

Contribution

It introduces a novel fuzzy pseudo-count approach that guides POMDP learning with limited data, reformulating it as a MAP estimation problem.

Findings

Outperforms standard EM in synthetic medical simulations

Successfully recovers a clinically coherent POMDP in a case study

Demonstrates robustness under low-data and high-noise conditions

Abstract

Learning the parameters of Partially Observable Markov Decision Processes (POMDPs) from limited data is a significant challenge. We introduce the Fuzzy MAP EM algorithm, a novel approach that incorporates expert knowledge into the parameter estimation process by enriching the Expectation Maximization (EM) framework with fuzzy pseudo-counts derived from an expert-defined fuzzy model. This integration naturally reformulates the problem as a Maximum A Posteriori (MAP) estimation, effectively guiding learning in environments with limited data. In synthetic medical simulations, our method consistently outperforms the standard EM algorithm under both low-data and high-noise conditions. Furthermore, a case study on Myasthenia Gravis illustrates the ability of the Fuzzy MAP EM algorithm to recover a clinically coherent POMDP, demonstrating its potential as a practical tool for data-efficient modeling in healthcare.