A Real-time Hand Gesture Recognition and Human-Computer Interaction System

TL;DR

This paper presents a real-time hand gesture recognition system using CNN and Kalman filter, enabling accurate, stable human-computer interaction with extendable command capabilities for diverse applications.

Contribution

The work introduces a CNN-based gesture recognition system combined with Kalman filtering for stable control, and a noise-reduction strategy to improve interaction reliability.

Findings

Achieved higher gesture recognition accuracy with monocular camera.

Enabled smooth mouse cursor control through Kalman filter estimation.

Enhanced system reliability by implementing noise filtering strategies.

Abstract

In this project, we design a real-time human-computer interaction system based on hand gesture. The whole system consists of three components: hand detection, gesture recognition and human-computer interaction (HCI) based on recognition; and realizes the robust control of mouse and keyboard events with a higher accuracy of gesture recognition. Specifically, we use the convolutional neural network (CNN) to recognize gestures and makes it attainable to identify relatively complex gestures using only one cheap monocular camera. We introduce the Kalman filter to estimate the hand position based on which the mouse cursor control is realized in a stable and smooth way. During the HCI stage, we develop a simple strategy to avoid the false recognition caused by noises - mostly transient, false gestures, and thus to improve the reliability of interaction. The developed system is highly extendable and can be used in human-robotic or other human-machine interaction scenarios with more complex command formats rather than just mouse and keyboard events.