Text writing in the air

TL;DR

This paper introduces a real-time air-writing system using a mobile camera that tracks colored fingertips and recognizes English characters via OCR, enabling natural interaction without additional hardware.

Contribution

It presents a novel, hardware-free air-writing method that combines fingertip tracking and OCR for text input on mobile devices.

Findings

Achieved an average accuracy of 92.08% in character recognition.

System is simple, fast, and does not require extra sensors or hardware.

Effective for disconnected languages with some color sensitivity limitations.

Abstract

This paper presents a real time video based pointing method which allows sketching and writing of English text over air in front of mobile camera. Proposed method have two main tasks: first it track the colored finger tip in the video frames and then apply English OCR over plotted images in order to recognize the written characters. Moreover, proposed method provides a natural human-system interaction in such way that it do not require keypad, stylus, pen or glove etc for character input. For the experiments, we have developed an application using OpenCv with JAVA language. We tested the proposed method on Samsung Galaxy3 android mobile. Results show that proposed algorithm gains the average accuracy of 92.083% when tested for different shaped alphabets. Here, more than 3000 different Magnetic 3D shaped characters were used [Ref: http://learnrnd.com/news.php?id=Magnetic_3D_Bio_Printing]. Our proposed system is the software based approach and relevantly very simple, fast and easy. It does not require sensors or any hardware rather than camera and red tape. Moreover, proposed methodology can be applicable for all disconnected languages but having one issue that it is color sensitive in such a way that existence of any red color in the background before starting the character writing can lead to false results.