Did somebody say "Gest-IT"? A pilot exploration of multimodal data management
Ludovica Pannitto, Lorenzo Albanesi, Laura Marion, Federica Maria Martines, Carmelo Caruso, Claudia S. Bianchini, Francesca Masini, Caterina Mauri

TL;DR
This paper explores the creation and analysis of a multimodal corpus with orthographic, prosodic, and gestural annotations to study gesture patterns in conversations involving sighted and visually impaired individuals.
Contribution
It introduces the Gest-IT resource, a multimodal corpus with a three-layer annotation scheme, and discusses transcription methods and future development steps.
Findings
Gesture patterns vary between sighted and visually impaired speakers
The Gest-IT corpus enables detailed multimodal analysis
A unified CoNLL-U format facilitates data sharing and analysis
Abstract
The paper presents a pilot exploration of the construction, management and analysis of a multimodal corpus. Through a three-layer annotation that provides orthographic, prosodic, and gestural transcriptions, the Gest-IT resource allows to investigate the variation of gesture-making patterns in conversations between sighted people and people with visual impairment. After discussing the transcription methods and technical procedures employed in our study, we propose a unified CoNLL-U corpus and indicate our future steps
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSemantic Web and Ontologies · linguistics and terminology studies · Digital Communication and Language
