HyReach: Vision-Guided Hybrid Manipulator Reaching in Unseen Cluttered Environments

TL;DR

This paper introduces a real-time hybrid rigid-soft manipulator system that combines vision-based perception and shape-aware planning for robust object reaching in unseen cluttered environments, achieving high accuracy without retraining.

Contribution

The work presents a novel hybrid manipulator system with integrated perception and planning that generalizes to new scenes without environment-specific retraining.

Findings

Achieves less than 2 cm reaching error in diverse cluttered environments.

Operates in real-time with no environment-specific retraining.

Demonstrates robustness and adaptability in unstructured scenes.

Abstract

As robotic systems increasingly operate in unstructured, cluttered, and previously unseen environments, there is a growing need for manipulators that combine compliance, adaptability, and precise control. This work presents a real-time hybrid rigid-soft continuum manipulator system designed for robust open-world object reaching in such challenging environments. The system integrates vision-based perception and 3D scene reconstruction with shape-aware motion planning to generate safe trajectories. A learning-based controller drives the hybrid arm to arbitrary target poses, leveraging the flexibility of the soft segment while maintaining the precision of the rigid segment. The system operates without environment-specific retraining, enabling direct generalization to new scenes. Extensive real-world experiments demonstrate consistent reaching performance with errors below 2 cm across diverse cluttered setups, highlighting the potential of hybrid manipulators for adaptive and reliable operation in unstructured environments.