Learning to Catch Piglets in Flight

TL;DR

This paper introduces a robotic system that combines RGB-D and radar sensors with deep learning to successfully catch flying objects, demonstrated on a stuffed Piglet with high accuracy.

Contribution

The paper presents a novel multi-sensor, deep learning-based control system for catching flying objects, outperforming traditional methods in accuracy.

Findings

Deep learning approach achieves 80% success rate in catching Piglet.

Multi-sensor fusion improves detection and interception accuracy.

Deep learning outperforms color filtering and ballistic regression methods.

Abstract

Catching objects in-flight is an outstanding challenge in robotics. In this paper, we present a closed-loop control system fusing data from two sensor modalities: an RGB-D camera and a radar. To develop and test our method, we start with an easy to identify object: a stuffed Piglet. We implement and compare two approaches to detect and track the object, and to predict the interception point. A baseline model uses colour filtering for locating the thrown object in the environment, while the interception point is predicted using a least squares regression over the physical ballistic trajectory equations. A deep learning based method uses artificial neural networks for both object detection and interception point prediction. We show that we are able to successfully catch Piglet in 80% of the cases with our deep learning approach.