DiffSRL: Learning Dynamical State Representation for Deformable Object Manipulation with Differentiable Simulator

TL;DR

DiffSRL introduces a differentiable simulation-based pipeline for learning dynamic state representations of deformable objects, improving long-term dynamics modeling and reward prediction in complex systems.

Contribution

It presents a novel end-to-end learning method that embeds complex deformable object dynamics into the training process using differentiable simulation.

Findings

Superior performance in capturing long-term dynamics

Effective reward prediction in deformable object simulation

Establishment of a new benchmark for state representation learning

Abstract

Dynamic state representation learning is an important task in robot learning. Latent space that can capture dynamics related information has wide application in areas such as accelerating model free reinforcement learning, closing the simulation to reality gap, as well as reducing the motion planning complexity. However, current dynamic state representation learning methods scale poorly on complex dynamic systems such as deformable objects, and cannot directly embed well defined simulation function into the training pipeline. We propose DiffSRL, a dynamic state representation learning pipeline utilizing differentiable simulation that can embed complex dynamics models as part of the end-to-end training. We also integrate differentiable dynamic constraints as part of the pipeline which provide incentives for the latent state to be aware of dynamical constraints. We further establish a state representation learning benchmark on a soft-body simulation system, PlasticineLab, and our model demonstrates superior performance in terms of capturing long-term dynamics as well as reward prediction.