SignCol: Open-Source Software for Collecting Sign Language Gestures

TL;DR

SignCol is an open-source tool that uses Microsoft Kinect to collect and store diverse sign language gestures, supporting multi-language datasets to aid recognition and assistive technologies.

Contribution

It introduces a comprehensive, multi-modal gesture capturing software tailored for sign language recognition research and multi-language dataset development.

Findings

Supports capturing RGB, depth, infrared, and skeleton data.

Enables creation of diverse sign language gesture datasets.

Provides statistical insights into recorded sign items.

Abstract

Sign(ed) languages use gestures, such as hand or head movements, for communication. Sign language recognition is an assistive technology for individuals with hearing disability and its goal is to improve such individuals' life quality by facilitating their social involvement. Since sign languages are vastly varied in alphabets, as known as signs, a sign recognition software should be capable of handling eight different types of sign combinations, e.g. numbers, letters, words and sentences. Due to the intrinsic complexity and diversity of symbolic gestures, recognition algorithms need a comprehensive visual dataset to learn by. In this paper, we describe the design and implementation of a Microsoft Kinect-based open source software, called SignCol, for capturing and saving the gestures used in sign languages. Our work supports a multi-language database and reports the recorded items statistics. SignCol can capture and store colored(RGB) frames, depth frames, infrared frames, body index frames, coordinate mapped color-body frames, skeleton information of each frame and camera parameters simultaneously.