Towards Ubiquitous Intelligent Hand Interaction

TL;DR

This paper proposes a comprehensive research framework for ubiquitous intelligent hand interaction, addressing limitations of current vision-based hand tracking systems in pervasive environments.

Contribution

It introduces a new framework analyzing sensors, behavior, and semantics to enable practical, ubiquitous hand interaction beyond traditional vision-based solutions.

Findings

Analysis of sensor, behavior, and semantic aspects for hand interaction

Summary of previous research in intelligent hand interaction

Future research directions outlined based on the framework

Abstract

The development of ubiquitous computing and sensing devices has brought about novel interaction scenarios such as mixed reality and IoT (e.g., smart home), which pose new demands for the next generation of natural user interfaces (NUI). Human hand, benefit for the large degree-of-freedom, serves as a medium through which people interact with the external world in their daily lives, thus also being regarded as the main entry of NUI. Unfortunately, current hand tracking system is largely confined on first perspective vision-based solutions, which suffer from optical artifacts and are not practical in ubiquitous environments. In my thesis, I rethink this problem by analyzing the underlying logic in terms of sensor, behavior, and semantics, constituting a research framework for achieving ubiquitous intelligent hand interaction. Then I summarize my previous research topics and illustrated the future research directions based on my research framework.