Dynamic Object Catching with Quadruped Robot Front Legs

TL;DR

This paper introduces a comprehensive framework enabling a quadruped robot to visually detect, predict, and catch moving objects using its front legs, combining computer vision, trajectory prediction, and precise leg control.

Contribution

It presents a novel integrated system for dynamic object catching with quadruped robots, including three methods for optimal catching point selection and validation through real-world experiments.

Findings

GMM-based method achieved 80% success rate

System successfully detects and intercepts thrown objects

Robust performance across various scenarios

Abstract

This paper presents a framework for dynamic object catching using a quadruped robot's front legs while it stands on its rear legs. The system integrates computer vision, trajectory prediction, and leg control to enable the quadruped to visually detect, track, and successfully catch a thrown object using an onboard camera. Leveraging a fine-tuned YOLOv8 model for object detection and a regression-based trajectory prediction module, the quadruped adapts its front leg positions iteratively to anticipate and intercept the object. The catching maneuver involves identifying the optimal catching position, controlling the front legs with Cartesian PD control, and closing the legs together at the right moment. We propose and validate three different methods for selecting the optimal catching position: 1) intersecting the predicted trajectory with a vertical plane, 2) selecting the point on the predicted trajectory with the minimal distance to the center of the robot's legs in their nominal position, and 3) selecting the point on the predicted trajectory with the highest likelihood on a Gaussian Mixture Model (GMM) modelling the robot's reachable space. Experimental results demonstrate robust catching capabilities across various scenarios, with the GMM method achieving the best performance, leading to an 80% catching success rate. A video demonstration of the system in action can be found at https://youtu.be/sm7RdxRfIYg .