Analyzing Neural Jacobian Methods in Applications of Visual Servoing and Kinematic Control

TL;DR

This paper investigates neural network approaches to approximate the Jacobian matrix for robotic control, demonstrating improved accuracy and conditioning in simulation and real-world experiments, facilitating adaptable control across different robots.

Contribution

It introduces two novel neural Jacobian learning methods for Cartesian control, one based on k-nearest neighbors and the other on differentiable neural kinematics, with empirical validation.

Findings

Both methods outperform existing data-driven approaches.

They produce more accurate and better-conditioned Jacobian matrices.

The methods are effective in real-world robotic experiments.

Abstract

Designing adaptable control laws that can transfer between different robots is a challenge because of kinematic and dynamic differences, as well as in scenarios where external sensors are used. In this work, we empirically investigate a neural networks ability to approximate the Jacobian matrix for an application in Cartesian control schemes. Specifically, we are interested in approximating the kinematic Jacobian, which arises from kinematic equations mapping a manipulator's joint angles to the end-effector's location. We propose two different approaches to learn the kinematic Jacobian. The first method arises from visual servoing where we learn the kinematic Jacobian as an approximate linear system of equations from the k-nearest neighbors for a desired joint configuration. The second, motivated by forward models in machine learning, learns the kinematic behavior directly and calculates the Jacobian by differentiating the learned neural kinematics model. Simulation experimental results show that both methods achieve better performance than alternative data-driven methods for control, provide closer approximations to the proper kinematics Jacobian matrix, and on average produce better-conditioned Jacobian matrices. Real-world experiments were conducted on a Kinova Gen-3 lightweight robotic manipulator, which includes an uncalibrated visual servoing experiment, a practical application of our methods, as well as a 7-DOF point-to-point task highlighting that our methods are applicable on real robotic manipulators.