Policy Space Identification in Configurable Environments
Alberto Maria Metelli, Guglielmo Manneschi, Marcello Restelli

TL;DR
This paper introduces statistical testing methods to identify the controllable policy parameters of an agent within configurable environments, enhancing understanding of agent capabilities through probabilistic analysis and empirical validation.
Contribution
It presents novel identification rules for policy space detection, including a probabilistic analysis for linear policies, and leverages environment configurability to improve identification accuracy.
Findings
Effective policy space identification in discrete and continuous domains
Probabilistic analysis validates the simplified identification rule
Configurable environments enhance parameter control detection
Abstract
We study the problem of identifying the policy space of a learning agent, having access to a set of demonstrations generated by its optimal policy. We introduce an approach based on statistical testing to identify the set of policy parameters the agent can control, within a larger parametric policy space. After presenting two identification rules (combinatorial and simplified), applicable under different assumptions on the policy space, we provide a probabilistic analysis of the simplified one in the case of linear policies belonging to the exponential family. To improve the performance of our identification rules, we frame the problem in the recently introduced framework of the Configurable Markov Decision Processes, exploiting the opportunity of configuring the environment to induce the agent revealing which parameters it can control. Finally, we provide an empirical evaluation, on…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
