What makes a gesture a gesture? Neural signatures involved in gesture recognition

TL;DR

This study investigates neural signatures of gesture recognition using EEG, finding that inflection points in gestures correlate with oscillatory activity in visual and motor cortices, supporting their role as placeholders in cognition.

Contribution

It validates the hypothesis that salient motion inflection points are linked to neural activity, bridging gesture kinematics with EEG signatures in gesture recognition.

Findings

Inflection points correlate with EEG mu rhythm peaks.

EEG power increases occur 380-500ms after inflection points.

Visual and motor cortex activity is sensitive to gesture motion trajectories.

Abstract

Previous work in the area of gesture production, has made the assumption that machines can replicate "human-like" gestures by connecting a bounded set of salient points in the motion trajectory. Those inflection points were hypothesized to also display cognitive saliency. The purpose of this paper is to validate that claim using electroencephalography (EEG). That is, this paper attempts to find neural signatures of gestures (also referred as placeholders) in human cognition, which facilitate the understanding, learning and repetition of gestures. Further, it is discussed whether there is a direct mapping between the placeholders and kinematic salient points in the gesture trajectories. These are expressed as relationships between inflection points in the gestures' trajectories with oscillatory mu rhythms (8-12 Hz) in the EEG. This is achieved by correlating fluctuations in mu power during gesture observation with salient motion points found for each gesture. Peaks in the EEG signal at central electrodes (motor cortex) and occipital electrodes (visual cortex) were used to isolate the salient events within each gesture. We found that a linear model predicting mu peaks from motion inflections fits the data well. Increases in EEG power were detected 380 and 500ms after inflection points at occipital and central electrodes, respectively. These results suggest that coordinated activity in visual and motor cortices is sensitive to motion trajectories during gesture observation, and it is consistent with the proposal that inflection points operate as placeholders in gesture recognition.