Towards Open-World Gesture Recognition

TL;DR

This paper addresses the challenge of adapting wrist-worn gesture recognition models to changing data distributions in real-world scenarios by proposing a continual learning approach and systematic offline analysis for open-world gesture recognition.

Contribution

It introduces the concept of open-world gesture recognition, applies continual learning to this problem, and provides a design engineering framework with guidelines for development.

Findings

Systematic offline analysis of continual learning methods

Design guidelines for open-world wrist gesture recognition

Improved adaptability to changing data distributions

Abstract

Providing users with accurate gestural interfaces, such as gesture recognition based on wrist-worn devices, is a key challenge in mixed reality. However, static machine learning processes in gesture recognition assume that training and test data come from the same underlying distribution. Unfortunately, in real-world applications involving gesture recognition, such as gesture recognition based on wrist-worn devices, the data distribution may change over time. We formulate this problem of adapting recognition models to new tasks, where new data patterns emerge, as open-world gesture recognition (OWGR). We propose the use of continual learning to enable machine learning models to be adaptive to new tasks without degrading performance on previously learned tasks. However, the process of exploring parameters for questions around when, and how, to train and deploy recognition models requires resource-intensive user studies may be impractical. To address this challenge, we propose a design engineering approach that enables offline analysis on a collected large-scale dataset by systematically examining various parameters and comparing different continual learning methods. Finally, we provide design guidelines to enhance the development of an open-world wrist-worn gesture recognition process.