Coherency in One-Shot Gesture Recognition

TL;DR

This paper introduces a framework for one-shot gesture recognition that generates realistic gesture samples from minimal data, and proposes a new coherency metric to evaluate the agreement between machine and human recognition.

Contribution

It presents a novel approach focusing on gesture generation from single observations using human interaction characteristics, and introduces a coherency metric for validation.

Findings

Achieved 89.2% recognition accuracy with classifiers

Human recognition of robot-performed gestures reached 92.5%

Proposed coherency metric showed 93.6% agreement between machine and human recognition

Abstract

User's intentions may be expressed through spontaneous gesturing, which have been seen only a few times or never before. Recognizing such gestures involves one shot gesture learning. While most research has focused on the recognition of the gestures itself, recently new approaches were proposed to deal with gesture perception and production as part of the same problem. The framework presented in this work focuses on learning the process that leads to gesture generation, rather than mining the gesture's associated features. This is achieved using kinematic, cognitive and biomechanic characteristics of human interaction. These factors enable the artificial production of realistic gesture samples originated from a single observation. The generated samples are then used as training sets for different state-of-the-art classifiers. Performance is obtained first, by observing the machines' gesture recognition percentages. Then, performance is computed by the human recognition from gestures performed by robots. Based on these two scenarios, a composite new metric of coherency is proposed relating to the amount of agreement between these two conditions. Experimental results provide an average recognition performance of 89.2% for the trained classifiers and 92.5% for the participants. Coherency in recognition was determined at 93.6%. While this new metric is not directly comparable to raw accuracy or other pure performance-based standard metrics, it provides a quantifier for validating how realistic the machine generated samples are and how accurate the resulting mimicry is.