Hierarchical Successor Representation for Robust Transfer
Changmin Yu, M\'at\'e Lengyel

TL;DR
This paper introduces Hierarchical Successor Representation (HSR), a novel approach that incorporates temporal abstractions to create stable, interpretable, and transfer-efficient predictive representations in complex environments.
Contribution
The paper proposes HSR, a hierarchical extension of SR that improves robustness to policy changes and environmental complexity, enabling better transfer and exploration.
Findings
HSR learns stable, policy-agnostic features.
NMF on HSR yields sparse, interpretable maps.
HSR enhances transfer and exploration in complex environments.
Abstract
The successor representation (SR) provides a powerful framework for decoupling predictive dynamics from rewards, enabling rapid generalisation across reward configurations. However, the classical SR is limited by its inherent policy dependence: policies change due to ongoing learning, environmental non-stationarities, and changes in task demands, making established predictive representations obsolete. Furthermore, in topologically complex environments, SRs suffer from spectral diffusion, leading to dense and overlapping features that scale poorly. Here we propose the Hierarchical Successor Representation (HSR) for overcoming these limitations. By incorporating temporal abstractions into the construction of predictive representations, HSR learns stable state features which are robust to task-induced policy changes. Applying non-negative matrix factorisation (NMF) to the HSR yields a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Reinforcement Learning in Robotics · Generative Adversarial Networks and Image Synthesis
