Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation

TL;DR

This paper introduces CAVIN Planner, a hierarchical, model-based approach using cascaded variational inference to generate multi-step manipulation plans, effectively modeling dynamics at different temporal levels for complex robotic tasks.

Contribution

It proposes a novel hierarchical planning method that learns decoupled latent representations for high-level effects and low-level motions, improving multi-step manipulation planning.

Findings

Outperforms state-of-the-art model-based methods in cluttered tabletop tasks

Effectively models dynamics at multiple temporal resolutions

Successfully handles high-dimensional observations in robotic manipulation

Abstract

The fundamental challenge of planning for multi-step manipulation is to find effective and plausible action sequences that lead to the task goal. We present Cascaded Variational Inference (CAVIN) Planner, a model-based method that hierarchically generates plans by sampling from latent spaces. To facilitate planning over long time horizons, our method learns latent representations that decouple the prediction of high-level effects from the generation of low-level motions through cascaded variational inference. This enables us to model dynamics at two different levels of temporal resolutions for hierarchical planning. We evaluate our approach in three multi-step robotic manipulation tasks in cluttered tabletop environments given high-dimensional observations. Empirical results demonstrate that the proposed method outperforms state-of-the-art model-based methods by strategically interacting with multiple objects.