Autonomous Field-of-View Adjustment Using Adaptive Kinematic Constrained Control with Robot-Held Microscopic Camera Feedback

TL;DR

This paper introduces an adaptive control method for robotic microscopic cameras that maintains the camera within its limited field-of-view during manipulation tasks, improving visibility and precision.

Contribution

It models camera extrinsics within the robot's kinematic framework and employs a U-Net based tool tracking for real-time adaptation during tasks.

Findings

Camera stayed within FoV 94.1% of the time with the proposed method.

Significant improvement over 54.4% FoV retention without adaptive control.

Validated in a bi-manual microscopic manipulation setup.

Abstract

Robotic systems for manipulation in millimeter scale often use a camera with high magnification for visual feedback of the target region. However, the limited field-of-view (FoV) of the microscopic camera necessitates camera motion to capture a broader workspace environment. In this work, we propose an autonomous robotic control method to constrain a robot-held camera within a designated FoV. Furthermore, we model the camera extrinsics as part of the kinematic model and use camera measurements coupled with a U-Net based tool tracking to adapt the complete robotic model during task execution. As a proof-of-concept demonstration, the proposed framework was evaluated in a bi-manual setup, where the microscopic camera was controlled to view a tool moving in a pre-defined trajectory. The proposed method allowed the camera to stay 94.1% of the time within the real FoV, compared to 54.4% without the proposed adaptive control.