Scheduled Intrinsic Drive: A Hierarchical Take on Intrinsically Motivated Exploration
Jingwei Zhang, Niklas Wetzel, Nicolai Dorka, Joschka Boedecker and, Wolfram Burgard

TL;DR
This paper introduces a hierarchical exploration method in reinforcement learning that separates intrinsic and extrinsic policies, using a novel successor feature control intrinsic reward to improve exploration efficiency in visual environments.
Contribution
It proposes a new scheduled intrinsic drive framework with separate policies and a task-agnostic intrinsic reward called successor feature control, enhancing exploration in sparse reward settings.
Findings
Significant improvement in exploration efficiency across three visual environments.
The hierarchical scheduling of intrinsic and extrinsic policies stabilizes learning.
Successor feature control provides a general intrinsic motivation signal.
Abstract
Exploration in sparse reward reinforcement learning remains an open challenge. Many state-of-the-art methods use intrinsic motivation to complement the sparse extrinsic reward signal, giving the agent more opportunities to receive feedback during exploration. Commonly these signals are added as bonus rewards, which results in a mixture policy that neither conducts exploration nor task fulfillment resolutely. In this paper, we instead learn separate intrinsic and extrinsic task policies and schedule between these different drives to accelerate exploration and stabilize learning. Moreover, we introduce a new type of intrinsic reward denoted as successor feature control (SFC), which is general and not task-specific. It takes into account statistics over complete trajectories and thus differs from previous methods that only use local information to evaluate intrinsic motivation. We evaluate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsReinforcement Learning in Robotics · Functional Brain Connectivity Studies · EEG and Brain-Computer Interfaces
