Smartphone Sensors for Modeling Human-Computer Interaction: General Outlook and Research Datasets for User Authentication

TL;DR

This paper reviews smartphone sensors for modeling human-computer interaction, categorizes applications, summarizes datasets for user authentication, and demonstrates a biometric system achieving up to 87% accuracy using touch gestures.

Contribution

It provides a comprehensive taxonomy of sensor-based HCI applications, summarizes key research datasets, and introduces a novel multimodal database with experimental validation for user authentication.

Findings

Achieved up to 87% accuracy in biometric authentication.

Presented a new multimodal mobile database with 14 sensors from 600 participants.

Demonstrated the effectiveness of simple touch gestures for user identification.

Abstract

In this paper we list the sensors commonly available in modern smartphones and provide a general outlook of the different ways these sensors can be used for modeling the interaction between human and smartphones. We then provide a taxonomy of applications that can exploit the signals originated by these sensors in three different dimensions, depending on the main information content embedded in the signals exploited in the application: neuromotor skills, cognitive functions, and behaviors/routines. We then summarize a representative selection of existing research datasets in this area, with special focus on applications related to user authentication, including key features and a selection of the main research results obtained on them so far. Then, we perform the experimental work using the HuMIdb database (Human Mobile Interaction database), a novel multimodal mobile database that includes 14 mobile sensors captured from 600 participants. We evaluate a biometric authentication system based on simple linear touch gestures using a Siamese Neural Network architecture. Very promising results are achieved with accuracies up to 87% for person authentication based on a simple and fast touch gesture.