Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators

TL;DR

This paper presents a CNN-based keypoint detection method to improve visual servoing of manipulators, replacing fiducial markers with more realistic object features, validated through real-world experiments and dataset augmentation.

Contribution

The paper introduces a novel CNN-based keypoint detection technique that enhances visual servoing accuracy by focusing on realistic object corners instead of fiducial markers.

Findings

50% reduction in validation loss with model modifications

Effective keypoint detection on real-world images

Improved convergence of visual servoing algorithms

Abstract

This paper introduces an innovative keypoint detection technique based on Convolutional Neural Networks (CNNs) to enhance the performance of existing Deep Visual Servoing (DVS) models. To validate the convergence of the Image-Based Visual Servoing (IBVS) algorithm, real-world experiments utilizing fiducial markers for feature detection are conducted before designing the CNN-based feature detector. To address the limitations of fiducial markers, the novel feature detector focuses on extracting keypoints that represent the corners of a more realistic object compared to fiducial markers. A dataset is generated from sample data captured by the camera mounted on the robot end-effector while the robot operates randomly in the task space. The samples are automatically labeled, and the dataset size is increased by flipping and rotation. The CNN model is developed by modifying the VGG-19 pre-trained on the ImageNet dataset. While the weights in the base model remain fixed, the fully connected layer's weights are updated to minimize the mean absolute error, defined based on the deviation of predictions from the real pixel coordinates of the corners. The model undergoes two modifications: replacing max-pooling with average-pooling in the base model and implementing an adaptive learning rate that decreases during epochs. These changes lead to a 50 percent reduction in validation loss. Finally, the trained model's reliability is assessed through k-fold cross-validation.