Robust Point Cloud Reinforcement Learning via PCA-Based Canonicalization

TL;DR

This paper introduces PCA Point Cloud (PPC), a canonicalization method that enhances the robustness of Point Cloud Reinforcement Learning to camera pose variations, improving reliability in robotic control tasks.

Contribution

The paper presents PPC, a novel canonicalization framework that maps point clouds to a unique pose, reducing viewpoint sensitivity in PC-RL for better robotic control.

Findings

PPC improves robustness to unseen camera poses.

PPC reduces viewpoint-induced inconsistencies.

Enhances reliability in robotic tasks.

Abstract

Reinforcement Learning (RL) from raw visual input has achieved impressive successes in recent years, yet it remains fragile to out-of-distribution variations such as changes in lighting, color, and viewpoint. Point Cloud Reinforcement Learning (PC-RL) offers a promising alternative by mitigating appearance-based brittleness, but its sensitivity to camera pose mismatches continues to undermine reliability in realistic settings. To address this challenge, we propose PCA Point Cloud (PPC), a canonicalization framework specifically tailored for downstream robotic control. PPC maps point clouds under arbitrary rigid-body transformations to a unique canonical pose, aligning observations to a consistent frame, thereby substantially decreasing viewpoint-induced inconsistencies. In our experiments, we show that PPC improves robustness to unseen camera poses across challenging robotic tasks, providing a principled alternative to domain randomization.