Vision-Guided Grasp Planning for Prosthetic Hands in Unstructured Environments

TL;DR

This paper introduces a real-time vision-guided grasp planning system for prosthetic hands that enables adaptive, object-specific grasping in unstructured environments through perception, planning, and control integration.

Contribution

It presents a novel modular pipeline combining BVH-based segmentation, RRT* trajectory planning, and DLS inverse kinematics for dexterous, adaptive grasping in prosthetic hands.

Findings

Validated in simulation and on a physical prosthetic platform.

Achieved real-time, object-specific grasp configurations.

Demonstrated effectiveness in unstructured environments.

Abstract

Recent advancements in prosthetic technology have increasingly focused on enhancing dexterity and autonomy through intelligent control systems. Vision-based approaches offer promising results for enabling prosthetic hands to interact more naturally with diverse objects in dynamic environments. Building on this foundation, the paper presents a vision-guided grasping algorithm for a prosthetic hand, integrating perception, planning, and control for dexterous manipulation. A camera mounted on the set up captures the scene, and a Bounding Volume Hierarchy (BVH)-based vision algorithm is employed to segment an object for grasping and define its bounding box. Grasp contact points are then computed by generating candidate trajectories using Rapidly-exploring Random Tree Star algorithm, and selecting fingertip end poses based on the minimum Euclidean distance between these trajectories and the objects point cloud. Each finger grasp pose is determined independently, enabling adaptive, object-specific configurations. Damped Least Square (DLS) based Inverse kinematics solver is used to compute the corresponding joint angles, which are subsequently transmitted to the finger actuators for execution. This modular pipeline enables per-finger grasp planning and supports real-time adaptability in unstructured environments. The proposed method is validated in simulation, and experimental integration on a Linker Hand O7 platform.