Eye-Tracking-Driven Control in Daily Task Assistance for Assistive Robotic Arms

TL;DR

This paper introduces an eye-tracking-driven control system for assistive robotic arms that enables users with severe disabilities to perform daily tasks independently, addressing accuracy and interpretation challenges in gaze-based control.

Contribution

It presents a novel framework combining task pictograms and feature matching for reliable object selection without requiring user position knowledge.

Findings

Correctly interpreted object and task selection in 97.9% of measurements

Identified and improved issues through lessons learned

Framework is adaptable to new tasks and objects

Abstract

Shared control improves Human-Robot Interaction by reducing the user's workload and increasing the robot's autonomy. It allows robots to perform tasks under the user's supervision. Current eye-tracking-driven approaches face several challenges. These include accuracy issues in 3D gaze estimation and difficulty interpreting gaze when differentiating between multiple tasks. We present an eye-tracking-driven control framework, aimed at enabling individuals with severe physical disabilities to perform daily tasks independently. Our system uses task pictograms as fiducial markers combined with a feature matching approach that transmits data of the selected object to accomplish necessary task related measurements with an eye-in-hand configuration. This eye-tracking control does not require knowledge of the user's position in relation to the object. The framework correctly interpreted object and task selection in up to 97.9% of measurements. Issues were found in the evaluation, that were improved and shared as lessons learned. The open-source framework can be adapted to new tasks and objects due to the integration of state-of-the-art object detection models.