Demonstration of real-time event camera to collaborative robot communication

TL;DR

This paper demonstrates that event cameras can be effectively used as a low-latency data source for real-time human-robot interaction, enabling rapid communication and control.

Contribution

It introduces a novel application of event cameras for real-time robot control, showing feasibility in a human-robot interaction scenario.

Findings

Total delay of 110 ms from hand movement to robot response

Event cameras provide high-frequency, lightweight data suitable for real-time control

Robot dynamics contribute significantly to overall latency

Abstract

Real-time robot actuation is one of the main challenges to overcome in human-robot interaction. Most visual sensors are either too slow or their data are too complex to provide meaningful information and low latency input to a robotic system. Data output of an event camera is high-frequency and extremely lightweight, with only 8 bytes per event. To evaluate the hypothesis of using event cameras as data source for a real-time robotic system, the position of a waving hand is acquired from the event data and transmitted to a collaborative robot as a movement command. A total time delay of 110 ms was measured between the original movement and the robot movement, where much of the delay is caused by the robot dynamics.