Leveraging CVAE for Joint Configuration Estimation of Multifingered Grippers from Point Cloud Data

TL;DR

This paper introduces a CVAE-based method to efficiently estimate the joint configuration of multifingered robotic grippers from point cloud data, outperforming traditional inverse kinematics in speed and maintaining high accuracy.

Contribution

The paper presents a novel machine learning approach using CVAE to directly infer joint configurations from point clouds, bypassing complex IK calculations and enabling rapid grasp planning.

Findings

Operates within 0.05 milliseconds per inference

Achieves accuracy comparable to state-of-the-art methods

Validated on the MultiDex dataset with Allegro Hand

Abstract

This paper presents an efficient approach for determining the joint configuration of a multifingered gripper solely from the point cloud data of its poly-articulated chain, as generated by visual sensors, simulations or even generative neural networks. Well-known inverse kinematics (IK) techniques can provide mathematically exact solutions (when they exist) for joint configuration determination based solely on the fingertip pose, but often require post-hoc decision-making by considering the positions of all intermediate phalanges in the gripper's fingers, or rely on algorithms to numerically approximate solutions for more complex kinematics. In contrast, our method leverages machine learning to implicitly overcome these challenges. This is achieved through a Conditional Variational Auto-Encoder (CVAE), which takes point cloud data of key structural elements as input and reconstructs the corresponding joint configurations. We validate our approach on the MultiDex grasping dataset using the Allegro Hand, operating within 0.05 milliseconds and achieving accuracy comparable to state-of-the-art methods. This highlights the effectiveness of our pipeline for joint configuration estimation within the broader context of AI-driven techniques for grasp planning.