Protocol for identifying shared articulatory features of gestures and LSF: application to epistemic gesture

TL;DR

This paper introduces a new methodology combining manual and semi-automatic annotation to analyze epistemic gestures in French speech and Sign Language, revealing specific kinesiological markers of certainty and uncertainty.

Contribution

It presents a novel combined annotation approach for analyzing head movements in epistemic gestures across speech and sign language.

Findings

Head nods passing through neutral position indicate certainty.

Head held away from neutral position indicates uncertainty.

High movement speed correlates with certainty, low speed with uncertainty.

Abstract

This article focuses on the articulatory characteristics of epistemic gestures (i.e., gestures used to express certainty or uncertainty) in co-speech gestures (CSG) in French and in French Sign Language (LSF). It presents a new methodology for analysis, which relies on the complementary use of manual annotation (using Typannot) and semi-automatic annotation (using AlphaPose) to highlight the kinesiological characteristics of these epistemic gestures. The presented methodology allows to analyze the flexion/extension movements of the head in epistemic contexts. The results of this analysis show that in CSG and LSF: (1) head nods passing through the neutral position (i.e., head straight with no flexion/extension) and high movement speed are markers of certainty; and (2) holding the head position away from the neutral position and low movement speed indicate uncertainty. This study is conducted within the framework of the ANR LexiKHuM project, which develops kinesthetic communication solutions for human-machine interaction.