Hilbert Space Embeddings of POMDPs
Yu Nishiyama, Abdeslam Boularias, Arthur Gretton, Kenji Fukumizu

TL;DR
This paper introduces a nonparametric method for policy learning in POMDPs using kernel embeddings of distributions, enabling effective estimation of policies and value functions in a reproducing kernel Hilbert space.
Contribution
It presents a novel kernel-based framework for POMDP policy learning, leveraging distribution embeddings and kernel Bayes' rule for improved estimation.
Findings
Successfully learned policies using the feature space representation
Demonstrated the effectiveness of the approach through experiments
Provided a new nonparametric method for POMDPs
Abstract
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' rule to these distribution embeddings. Policies and value functions are defined on the feature space over states, which leads to a feature space expression for the Bellman equation. Value iteration may then be used to estimate the optimal value function and associated policy. Experimental results confirm that the correct policy is learned using the feature space representation.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsWater Systems and Optimization · Machine Learning and Algorithms · Network Security and Intrusion Detection
