# IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL

## TL;DR

This paper introduces IR-VIC, an unsupervised method for discovering sub-goals in reinforcement learning that enhances exploration and reduces sample complexity in partially observable environments.

## Contribution

It presents a novel framework combining variational intrinsic control with information-theoretic regularization to identify useful sub-goals without explicit supervision.

## Key findings

- Sub-goals improve exploration efficiency.
- Method outperforms supervised approaches in grid-world tasks.
- Enhances sample efficiency in partial observability scenarios.

## Abstract

We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our sub-goals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.