A multi-modal table tennis robot system

TL;DR

This paper presents an advanced multi-modal table tennis robot system integrating high-precision vision detection, novel calibration, and spin estimation techniques, enabling faster and more accurate robot responses in dynamic gameplay.

Contribution

It introduces a novel calibration method, an improved spin estimation approach, and combines event-based camera data with SNNs for enhanced ball detection in robotic table tennis.

Findings

High accuracy vision detection achieved

Improved spin estimation method developed

Event-based camera and SNN used for accurate ball detection

Abstract

In recent years, robotic table tennis has become a popular research challenge for perception and robot control. Here, we present an improved table tennis robot system with high accuracy vision detection and fast robot reaction. Based on previous work, our system contains a KUKA robot arm with 6 DOF, with four frame-based cameras and two additional event-based cameras. We developed a novel calibration approach to calibrate this multimodal perception system. For table tennis, spin estimation is crucial. Therefore, we introduced a novel, and more accurate spin estimation approach. Finally, we show how combining the output of an event-based camera and a Spiking Neural Network (SNN) can be used for accurate ball detection.