Intrinsic Motivation for Encouraging Synergistic Behavior
Rohan Chitnis, Shubham Tulsiani, Saurabh Gupta, Abhinav Gupta

TL;DR
This paper introduces an intrinsic motivation mechanism for reinforcement learning that encourages agents to take actions leading to effects that cannot be predicted by individual agents alone, improving learning in cooperative tasks.
Contribution
It proposes a novel intrinsic motivation approach based on synergistic effects, validated in robotic and multi-agent tasks, outperforming traditional surprise-based methods.
Findings
Enhanced learning efficiency in cooperative tasks
Intrinsic motivation based on synergistic effects outperforms surprise-based methods
Effective in robotic manipulation and locomotion tasks
Abstract
We study the role of intrinsic motivation as an exploration bias for reinforcement learning in sparse-reward synergistic tasks, which are tasks where multiple agents must work together to achieve a goal they could not individually. Our key idea is that a good guiding principle for intrinsic motivation in synergistic tasks is to take actions which affect the world in ways that would not be achieved if the agents were acting on their own. Thus, we propose to incentivize agents to take (joint) actions whose effects cannot be predicted via a composition of the predicted effect for each individual agent. We study two instantiations of this idea, one based on the true states encountered, and another based on a dynamics model trained concurrently with the policy. While the former is simpler, the latter has the benefit of being analytically differentiable with respect to the action taken. We…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Robot Manipulation and Learning · Neural dynamics and brain function
