Online interpretation of numeric sign language using 2-d skeletal model

TL;DR

This paper presents an online system that recognizes numeric sign language gestures using a 2D skeletal model derived from real-time hand shape analysis via a web camera.

Contribution

It introduces a novel real-time recognition scheme for numeric sign language using a 2D skeletal model based on shape, orientation, and movement analysis.

Findings

Effective real-time recognition of 10 numeric signs

Accurate interpretation based on hand shape and skeletal modeling

System operates using standard web camera input

Abstract

Gesturing is one of the natural modes of human communication. Signs produced by gestures can have a basic meaning coupled with additional information that is layered over the basic meaning of the sign. Sign language is an important example of communicative gestures that are highly structured and well accepted across the world as a communication medium for deaf and dumb. In this paper, an online recognition scheme is proposed to interpret the standard numeric sign language comprising of 10 basic hand symbols. A web camera is used to capture the real time hand movements as input to the system. The basic meaning of the hand gesture is extracted from the input data frame by analysing the shape of the hand, considering its orientation, movement and location to be fixed. The input hand shape is processed to identify the palm structure, fingertips and their relative positions and the presence of the extended thumb. A 2-dimensional skeletal model is generated from the acquired shape information to represent and subsequently interpret the basic meaning of the hand gesture.