Spatial Relationship Based Features for Indian Sign Language Recognition

TL;DR

This paper introduces a novel spatial feature extraction method for Indian Sign Language recognition, utilizing face and hand component analysis, interval symbolic data, and a nearest neighbor classifier, achieving promising results on a large dataset.

Contribution

It proposes a new spatial feature extraction technique using local and global centroids and symbolic data for sign language recognition, addressing variability among signers.

Findings

Effective recognition accuracy demonstrated on a large sign database.

Spatial features outperform traditional methods in sign differentiation.

Symbolic similarity measures improve matching robustness.

Abstract

In this paper, the task of recognizing signs made by hearing impaired people at sentence level has been addressed. A novel method of extracting spatial features to capture hand movements of a signer has been proposed. Frames of a given video of a sign are preprocessed to extract face and hand components of a signer. The local centroids of the extracted components along with the global centroid are exploited to extract spatial features. The concept of interval valued type symbolic data has been explored to capture variations in the same sign made by the different signers at different instances of time. A suitable symbolic similarity measure is studied to establish matching between test and reference signs and a simple nearest neighbour classifier is used to recognize an unknown sign as one among the known signs by specifying a desired level of threshold. An extensive experimentation is conducted on a considerably large database of signs created by us during the course of research work in order to evaluate the performance of the proposed system