A Vision-Based Shared-Control Teleoperation Scheme for Controlling the Robotic Arm of a Four-Legged Robot

TL;DR

This paper introduces a vision-based teleoperation system for quadruped robots with robotic arms, enabling intuitive, safe, and real-time control through wrist pose estimation and collision avoidance, validated on real hardware.

Contribution

It presents a novel vision-based control scheme that simplifies teleoperation of quadruped robot arms with collision prevention, enhancing safety and usability in hazardous environments.

Findings

System achieves real-time, robust control of robotic arm via vision-based wrist tracking.

Collision detection and avoidance improve safety during teleoperation.

Validated system demonstrates effectiveness on a real quadruped robot.

Abstract

In hazardous and remote environments, robotic systems perform critical tasks demanding improved safety and efficiency. Among these, quadruped robots with manipulator arms offer mobility and versatility for complex operations. However, teleoperating quadruped robots is challenging due to the lack of integrated obstacle detection and intuitive control methods for the robotic arm, increasing collision risks in confined or dynamically changing workspaces. Teleoperation via joysticks or pads can be non-intuitive and demands a high level of expertise due to its complexity, culminating in a high cognitive load on the operator. To address this challenge, a teleoperation approach that directly maps human arm movements to the robotic manipulator offers a simpler and more accessible solution. This work proposes an intuitive remote control by leveraging a vision-based pose estimation pipeline that utilizes an external camera with a machine learning-based model to detect the operator's wrist position. The system maps these wrist movements into robotic arm commands to control the robot's arm in real-time. A trajectory planner ensures safe teleoperation by detecting and preventing collisions with both obstacles and the robotic arm itself. The system was validated on the real robot, demonstrating robust performance in real-time control. This teleoperation approach provides a cost-effective solution for industrial applications where safety, precision, and ease of use are paramount, ensuring reliable and intuitive robotic control in high-risk environments.