Exploratory Design of a Hands-free Video Game Controller for a Quadriplegic Individual

TL;DR

This paper presents a novel hands-free video game control method using real-time facial expression recognition, designed to assist quadriplegic individuals in gaming and virtual experiences.

Contribution

It introduces a new facial expression-based control system specifically designed for quadriplegic users, tested in collaboration with a patient, expanding assistive technology options.

Findings

Facial expression recognition can effectively control game actions.

Preliminary tests show potential for enabling complex virtual activities.

System demonstrates feasibility for assistive gaming applications.

Abstract

From colored pixels to hyper-realistic 3D landscapes of virtual reality, video games have evolved immensely over the last few decades. However, video game input still requires two-handed dexterous finger manipulations for simultaneous joystick and trigger or mouse and keyboard presses. In this work, we explore the design of a hands-free game control method using realtime facial expression recognition for individuals with neurological and neuromuscular diseases who are unable to use traditional game controllers. Similar to other Assistive Technologies (AT), our facial input technique is also designed and tested in collaboration with a graduate student who has Spinal Muscular Atrophy. Our preliminary evaluation shows the potential of facial expression recognition for augmenting the lives of quadriplegic individuals by enabling them to accomplish things like walking, running, flying or other adventures that may not be so attainable otherwise.