Effective Gesture Based Framework for Capturing User Input

TL;DR

This paper presents a novel gesture-based framework using image processing to create a virtual keyboard and mouse, enhancing accessibility and reducing hardware costs and waste.

Contribution

It introduces a new image processing framework for gesture recognition that enables virtual keyboard and mouse functionalities using a camera.

Findings

High accuracy in gesture detection

Cost-effective and sustainable solution

Improved accessibility for users with disabilities

Abstract

Computers today aren't just confined to laptops and desktops. Mobile gadgets like mobile phones and laptops also make use of it. However, one input device that hasn't changed in the last 50 years is the QWERTY keyboard. Users of virtual keyboards can type on any surface as if it were a keyboard thanks to sensor technology and artificial intelligence. In this research, we use the idea of image processing to create an application for seeing a computer keyboard using a novel framework which can detect hand gestures with precise accuracy while also being sustainable and financially viable. A camera is used to capture keyboard images and finger movements which subsequently acts as a virtual keyboard. In addition, a visible virtual mouse that accepts finger coordinates as input is also described in this study. This system has a direct benefit of reducing peripheral cost, reducing electronics waste generated due to external devices and providing accessibility to people who cannot use the traditional keyboard and mouse.