PointPatchRL -- Masked Reconstruction Improves Reinforcement Learning on Point Clouds

TL;DR

PointPatchRL introduces a transformer-based approach with masked reconstruction for reinforcement learning on point clouds, significantly enhancing performance in complex manipulation tasks involving deformable objects and geometric variations.

Contribution

The paper presents PointPatchRL, a novel transformer-based method with masked reconstruction for RL on point clouds, addressing previous limitations of simple encoder architectures.

Findings

Outperforms previous point-cloud RL architectures.

Achieves better results on complex manipulation tasks.

Effective on deformable objects and geometric variations.

Abstract

Perceiving the environment via cameras is crucial for Reinforcement Learning (RL) in robotics. While images are a convenient form of representation, they often complicate extracting important geometric details, especially with varying geometries or deformable objects. In contrast, point clouds naturally represent this geometry and easily integrate color and positional data from multiple camera views. However, while deep learning on point clouds has seen many recent successes, RL on point clouds is under-researched, with only the simplest encoder architecture considered in the literature. We introduce PointPatchRL (PPRL), a method for RL on point clouds that builds on the common paradigm of dividing point clouds into overlapping patches, tokenizing them, and processing the tokens with transformers. PPRL provides significant improvements compared with other point-cloud processing architectures previously used for RL. We then complement PPRL with masked reconstruction for representation learning and show that our method outperforms strong model-free and model-based baselines on image observations in complex manipulation tasks containing deformable objects and variations in target object geometry. Videos and code are available at https://alrhub.github.io/pprl-website