Information-Theoretic Policy Learning from Partial Observations with Fully Informed Decision Makers

TL;DR

This paper extends imitation from observations to feature-only demonstrations, developing information-theoretic methods to learn policies from partial observations using behavioral cloning, with connections to entropy-regularized MDPs.

Contribution

It introduces a novel approach for policy learning from limited feature observations, expanding the scope of imitation learning with an information-theoretic framework.

Findings

Effective policy extraction from feature-only demonstrations

Connections established with entropy-regularized MDPs

Potential for improved imitation learning with partial data

Abstract

In this work we formulate and treat an extension of the Imitation from Observations problem. Imitation from Observations is a generalisation of the well-known Imitation Learning problem where state-only demonstrations are considered. In our treatment we extend the scope of Imitation from Observations to feature-only demonstrations which could arguably be described as partial observations. Therewith we mean that the full state of the decision makers is unknown and imitation must take place on the basis of a limited set of features. We set out for methods that extract an executable policy directly from those features which, in the literature, would be referred to as Behavioural Cloning methods. Our treatment combines elements from probability and information theory and draws connections with entropy regularized Markov Decision Processes.