Mobile Exergames: Activity Recognition Based on Smartphone Sensors

TL;DR

This paper presents a smartphone-based activity recognition system integrated into an exergame, using sensors and machine learning to detect activities and voice commands, enhancing gameplay and potential applications.

Contribution

It introduces a novel activity recognition system combining sensor data and voice recognition within a game, demonstrating high accuracy and immersive interaction.

Findings

High recognition accuracy for human activities

Successful integration of voice commands with sensor-based detection

Enhanced user engagement through multimodal interaction

Abstract

Smartphone sensors can be extremely useful in providing information on the activities and behaviors of persons. Human activity recognition is increasingly used for games, medical, or surveillance. In this paper, we propose a proof-of-concept 2D endless game called Duck Catch & Fit, which implements a detailed activity recognition system that uses a smartphone accelerometer, gyroscope, and magnetometer sensors. The system applies feature extraction and learning mechanism to detect human activities like staying, side movements, and fake side movements. In addition, a voice recognition system is combined to recognize the word "fire" and raise the game's complexity. The results show that it is possible to use machine learning techniques to recognize human activity with high recognition levels. Also, the combination of movement-based and voice-based integrations contributes to a more immersive gameplay.