CLAMGen: Closed-Loop Arm Motion Generation via Multi-view Vision-Based RL

TL;DR

This paper introduces CLAMGen, a vision-based reinforcement learning approach for closed-loop arm trajectory generation that improves obstacle avoidance and sample efficiency using multi-view images and residual learning techniques.

Contribution

It presents a residual-RL method combining a goal-reaching policy with residual learning from images, enhancing collision avoidance and 3D understanding in robotic arm control.

Findings

Achieves higher success rates than RL baselines.

Improves sample efficiency and obstacle avoidance.

Enhances 3D perception from multi-view images.

Abstract

We propose a vision-based reinforcement learning (RL) approach for closed-loop trajectory generation in an arm reaching problem. Arm trajectory generation is a fundamental robotics problem which entails finding collision-free paths to move the robot's body (e.g. arm) in order to satisfy a goal (e.g. place end-effector at a point). While classical methods typically require the model of the environment to solve a planning, search or optimization problem, learning-based approaches hold the promise of directly mapping from observations to robot actions. However, learning a collision-avoidance policy using RL remains a challenge for various reasons, including, but not limited to, partial observability, poor exploration, low sample efficiency, and learning instabilities. To address these challenges, we present a residual-RL method that leverages a greedy goal-reaching RL policy as the base to improve exploration, and the base policy is augmented with residual state-action values and residual actions learned from images to avoid obstacles. Further more, we introduce novel learning objectives and techniques to improve 3D understanding from multiple image views and sample efficiency of our algorithm. Compared to RL baselines, our method achieves superior performance in terms of success rate.