American Sign Language Identification Using Hand Trackpoint Analysis

TL;DR

This paper presents a machine learning pipeline that uses hand track point coordinates to identify American Sign Language gestures with high accuracy, addressing challenges in computer vision for sign language recognition.

Contribution

It introduces a novel approach combining hand track point analysis with optimized pre-processing and machine learning algorithms for sign language identification.

Findings

Achieved 95.66% accuracy in sign language recognition

Compared 28 pre-processing techniques with three algorithms

Identified the most effective pre-processing and algorithm combination

Abstract

Sign Language helps people with Speaking and Hearing Disabilities communicate with others efficiently. Sign Language identification is a challenging area in the field of computer vision and recent developments have been able to achieve near perfect results for the task, though some challenges are yet to be solved. In this paper we propose a novel machine learning based pipeline for American Sign Language identification using hand track points. We convert a hand gesture into a series of hand track point coordinates that serve as an input to our system. In order to make the solution more efficient, we experimented with 28 different combinations of pre-processing techniques, each run on three different machine learning algorithms namely k-Nearest Neighbours, Random Forests and a Neural Network. Their performance was contrasted to determine the best pre-processing scheme and algorithm pair. Our system achieved an Accuracy of 95.66% to identify American sign language gestures.