CNS: Correspondence Encoded Neural Image Servo Policy

TL;DR

This paper introduces CNS, a graph neural network-based image servo policy that encodes keypoints and correspondence for high-precision, robust robotic positioning, demonstrating superior generalization and efficiency in simulation and real-world tests.

Contribution

The paper proposes a novel graph neural network architecture encoding keypoints and correspondence for image servo, enhancing generalization, precision, and efficiency in robotic control.

Findings

Achieves <0.3° and sub-millimeter accuracy

Operates at up to 40fps in real-time

Generalizes from simulation to real-world without fine-tuning

Abstract

Image servo is an indispensable technique in robotic applications that helps to achieve high precision positioning. The intermediate representation of image servo policy is important to sensor input abstraction and policy output guidance. Classical approaches achieve high precision but require clean keypoint correspondence, and suffer from limited convergence basin or weak feature error robustness. Recent learning-based methods achieve moderate precision and large convergence basin on specific scenes but face issues when generalizing to novel environments. In this paper, we encode keypoints and correspondence into a graph and use graph neural network as architecture of controller. This design utilizes both advantages: generalizable intermediate representation from keypoint correspondence and strong modeling ability from neural network. Other techniques including realistic data generation, feature clustering and distance decoupling are proposed to further improve efficiency, precision and generalization. Experiments in simulation and real-world verify the effectiveness of our method in speed (maximum 40fps along with observer), precision (<0.3{\deg} and sub-millimeter accuracy) and generalization (sim-to-real without fine-tuning). Project homepage (full paper with supplementary text, video and code): https://hhcaz.github.io/CNS-home