# Learning to Plan Hierarchically from Curriculum

## TL;DR

This paper introduces a hierarchical planning framework that learns abstract skills and transition dynamics to improve planning efficiency and generalization in complex, stochastic domains, validated through simulations and robotic experiments.

## Contribution

It presents a novel method for learning hierarchical skills and transition models simultaneously during planning, enhancing performance in large and stochastic environments.

## Key findings

- Hierarchical planning outperforms non-hierarchical methods in complex domains.
- Learned skills and dynamics generalize well to larger state spaces.
- Successful transfer from simulation to real robotic manipulation.

## Abstract

We present a framework for learning to plan hierarchically in domains with unknown dynamics. We enhance planning performance by exploiting problem structure in several ways: (i) We simplify the search over plans by leveraging knowledge of skill objectives, (ii) Shorter plans are generated by enforcing aggressively hierarchical planning, (iii) We learn transition dynamics with sparse local models for better generalisation. Our framework decomposes transition dynamics into skill effects and success conditions, which allows fast planning by reasoning on effects, while learning conditions from interactions with the world. We propose a simple method for learning new abstract skills, using successful trajectories stemming from completing the goals of a curriculum. Learned skills are then refined to leverage other abstract skills and enhance subsequent planning. We show that both conditions and abstract skills can be learned simultaneously while planning, even in stochastic domains. Our method is validated in experiments of increasing complexity, with up to 2^100 states, showing superior planning to classic non-hierarchical planners or reinforcement learning methods. Applicability to real-world problems is demonstrated in a simulation-to-real transfer experiment on a robotic manipulator.