Sign Language Recognition System using TensorFlow Object Detection API

TL;DR

This paper presents a real-time Indian Sign Language recognition system using TensorFlow's Object Detection API, leveraging transfer learning on a custom webcam dataset to facilitate communication with deaf and dumb individuals.

Contribution

It introduces a real-time sign language recognition model trained with transfer learning on a custom dataset, addressing limitations of previous non-real-time systems.

Findings

Achieved high accuracy with limited dataset size

Developed a real-time sign language recognition system

Utilized TensorFlow Object Detection API for effective detection

Abstract

Communication is defined as the act of sharing or exchanging information, ideas or feelings. To establish communication between two people, both of them are required to have knowledge and understanding of a common language. But in the case of deaf and dumb people, the means of communication are different. Deaf is the inability to hear and dumb is the inability to speak. They communicate using sign language among themselves and with normal people but normal people do not take seriously the importance of sign language. Not everyone possesses the knowledge and understanding of sign language which makes communication difficult between a normal person and a deaf and dumb person. To overcome this barrier, one can build a model based on machine learning. A model can be trained to recognize different gestures of sign language and translate them into English. This will help a lot of people in communicating and conversing with deaf and dumb people. The existing Indian Sing Language Recognition systems are designed using machine learning algorithms with single and double-handed gestures but they are not real-time. In this paper, we propose a method to create an Indian Sign Language dataset using a webcam and then using transfer learning, train a TensorFlow model to create a real-time Sign Language Recognition system. The system achieves a good level of accuracy even with a limited size dataset.