Feature Dynamic Bayesian Networks

TL;DR

This paper introduces PhiDBN, an extension of PhiMDPs using Dynamic Bayesian Networks, with a new cost criterion for automatic feature extraction to improve learning in complex environments.

Contribution

It develops a cost-based method for automatic feature selection in structured MDPs, enabling more effective modeling of large-scale environments.

Findings

Derived a cost criterion for feature relevance

Proposed a complete learning algorithm framework

Enhanced environment modeling with PhiDBN

Abstract

Feature Markov Decision Processes (PhiMDPs) are well-suited for learning agents in general environments. Nevertheless, unstructured (Phi)MDPs are limited to relatively simple environments. Structured MDPs like Dynamic Bayesian Networks (DBNs) are used for large-scale real-world problems. In this article I extend PhiMDP to PhiDBN. The primary contribution is to derive a cost criterion that allows to automatically extract the most relevant features from the environment, leading to the "best" DBN representation. I discuss all building blocks required for a complete general learning algorithm.