Efficient Empowerment
Maximilian Karl, Justin Bayer, Patrick van der Smagt

TL;DR
This paper introduces an efficient approximation method for calculating empowerment in continuous environments, enabling its application in complex, real-world scenarios like robotics.
Contribution
It proposes a novel, computationally efficient approximation for empowerment in continuous spaces, overcoming previous limitations of brute-force methods.
Findings
The method enables fast empowerment evaluation in continuous environments.
Application demonstrated on a pendulum swing-up task.
Facilitates future use in real-world robotics scenarios.
Abstract
Empowerment quantifies the influence an agent has on its environment. This is formally achieved by the maximum of the expected KL-divergence between the distribution of the successor state conditioned on a specific action and a distribution where the actions are marginalised out. This is a natural candidate for an intrinsic reward signal in the context of reinforcement learning: the agent will place itself in a situation where its action have maximum stability and maximum influence on the future. The limiting factor so far has been the computational complexity of the method: the only way of calculation has so far been a brute force algorithm, reducing the applicability of the method to environments with a small set discrete states. In this work, we propose to use an efficient approximation for marginalising out the actions in the case of continuous environments. This allows fast…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Gene Regulatory Network Analysis · Receptor Mechanisms and Signaling
