Inverse Kinematics with Vision-Based Constraints

TL;DR

This paper formulates the Visual Inverse Kinematics problem incorporating vision-based constraints, proposing a novel SDP-based solution validated through simulations to enhance robot control with visual feedback.

Contribution

It introduces the VIK problem with FoV constraints and develops a semidefinite programming approach to solve it, bridging IK and visual servo control.

Findings

Effective SDP-based method for VIK validated in simulations.

Incorporation of vision constraints improves robot configuration accuracy.

New cost functions tailored for vision objectives.

Abstract

This paper introduces the Visual Inverse Kinematics problem (VIK) to fill the gap between robot Inverse Kinematics (IK) and visual servo control. Different from the IK problem, the VIK problem seeks to find robot configurations subject to vision-based constraints, in addition to kinematic constraints. In this work, we develop a formulation of the VIK problem with a Field of View (FoV) constraint, enforcing the visibility of an object from a camera on the robot. Our proposed solution is based on the idea of adding a virtual kinematic chain connecting the physical robot and the object; the FoV constraint is then equivalent to a joint angle kinematic constraint. Along the way, we introduce multiple vision-based cost functions to fulfill different objectives. We solve this formulation of the VIK problem using a method that involves a semidefinite program (SDP) constraint followed by a rank minimization algorithm. The performance of this method for solving the VIK problem is validated through simulations.