# GECO: A Real-Time Computer Vision-Assisted Gesture Controller for Advanced IoT Home System

## TL;DR

GECO is a real-time gesture control system for smart homes that uses computer vision to let users control devices with hand gestures, offering low latency and high user satisfaction.

## Contribution

GECO introduces a markerless, on-device gesture controller for IoT systems with ultra-low latency and high intuitiveness for diverse users.

## Key findings

- GECO achieved ultra-low latency (<50 ms) across multiple Android devices.
- User testing showed high intuitiveness (9.5/10) and gesture accuracy (9.2/10).
- The system was perceived as inclusive for users with speech impairments and low tech literacy.

## Abstract

This paper introduces GECO, a real-time, computer vision-assisted gesture controller for IoT-based smart home systems. The platform uses a markerless MediaPipe interface that combines gesture-driven navigation and command execution, enabling intuitive control of multiple domestic devices. The system integrates binary and analog gestures, such as continuous light dimming based on thumb–index angles, while operating on-device through a private MQTT network. Technical evaluations across multiple Android devices have demonstrated ultra-low latency times (<50 ms), enabling real-time responsiveness. A user experience study with seventeen participants reported high intuitiveness (9.5/10), gesture accuracy (9.2/10), and perceived inclusivity, mainly for individuals with speech impairments and low technological literacy. These results position GECO as a lightweight, accessible, and privacy-preserving interaction framework, advancing the integration of artificial intelligence and IoT within smart home environments.