Image-to-Joint Inverse Kinematic of a Supportive Continuum Arm Using Deep Learning

TL;DR

This paper presents a deep learning approach using a modified VGG-16 network to estimate the joint variables of a continuum arm from images, offering real-time inverse kinematics with robustness to noise and occlusion.

Contribution

It introduces an image-to-joint neural network for inverse kinematics of a continuum arm, leveraging transfer learning and robustness to real-world image variations.

Findings

Achieved accurate joint estimation from images under various conditions

Demonstrated real-time inverse kinematics alternative to analytical models

Provided a publicly available dataset for further research

Abstract

In this work, a deep learning-based technique is used to study the image-to-joint inverse kinematics of a tendon-driven supportive continuum arm. An eye-off-hand configuration is considered by mounting a camera at a fixed pose with respect to the inertial frame attached at the arm base. This camera captures an image for each distinct joint variable at each sampling time to construct the training dataset. This dataset is then employed to adapt a feed-forward deep convolutional neural network, namely the modified VGG-16 model, to estimate the joint variable. One thousand images are recorded to train the deep network, and transfer learning and fine-tuning techniques are applied to the modified VGG-16 to further improve the training. Finally, training is also completed with a larger dataset of images that are affected by various types of noises, changes in illumination, and partial occlusion. The main contribution of this research is the development of an image-to-joint network that can estimate the joint variable given an image of the arm, even if the image is not captured in an ideal condition. The key benefits of this research are twofold: 1) image-to-joint mapping can offer a real-time alternative to computationally complex inverse kinematic mapping through analytical models; and 2) the proposed technique can provide robustness against noise, occlusion, and changes in illumination. The dataset is publicly available on Kaggle.