Eye Gaze Controlled Robotic Arm for Persons with SSMI

TL;DR

This paper presents an affordable, webcam-based eye gaze control system integrated with augmented reality to assist individuals with severe speech and motor impairments in controlling a robotic arm for tasks like fabric printing.

Contribution

It introduces a novel, low-cost eye gaze tracking interface combined with AR for robotic control, validated through user studies with both able-bodied users and SSMI individuals.

Findings

Able-bodied users select screen regions in under 2 seconds.

Users with SSMI select regions in median 4 seconds.

Robotic arm task completion times under 15 seconds for SSMI users.

Abstract

Background: People with severe speech and motor impairment (SSMI) often uses a technique called eye pointing to communicate with outside world. One of their parents, caretakers or teachers hold a printed board in front of them and by analyzing their eye gaze manually, their intentions are interpreted. This technique is often error prone and time consuming and depends on a single caretaker. Objective: We aimed to automate the eye tracking process electronically by using commercially available tablet, computer or laptop and without requiring any dedicated hardware for eye gaze tracking. The eye gaze tracker is used to develop a video see through based AR (augmented reality) display that controls a robotic device with eye gaze and deployed for a fabric printing task. Methodology: We undertook a user centred design process and separately evaluated the web cam based gaze tracker and the video see through based human robot interaction involving users with SSMI. We also reported a user study on manipulating a robotic arm with webcam based eye gaze tracker. Results: Using our bespoke eye gaze controlled interface, able bodied users can select one of nine regions of screen at a median of less than 2 secs and users with SSMI can do so at a median of 4 secs. Using the eye gaze controlled human-robot AR display, users with SSMI could undertake representative pick and drop task at an average duration less than 15 secs and reach a randomly designated target within 60 secs using a COTS eye tracker and at an average time of 2 mins using the webcam based eye gaze tracker.