How Pragmatics Shape Articulation: A Computational Case Study in STEM ASL Discourse

TL;DR

This study investigates how pragmatic factors influence sign articulation in ASL STEM discourse, revealing that dialogue context significantly alters sign duration and entrainment, informing sign language technology development.

Contribution

It introduces a novel motion capture dataset of ASL STEM dialogue and analyzes how pragmatics affect sign articulation and entrainment, bridging linguistic analysis and computational modeling.

Findings

Dialogue signs are 24.6%-44.6% shorter than isolated signs.

Sign duration reductions are absent in monologue contexts.

Sign embedding models can recognize STEM signs and measure entrainment.

Abstract

Most state-of-the-art sign language models are trained on interpreter or isolated vocabulary data, which overlooks the variability that characterizes natural dialogue. However, human communication dynamically adapts to contexts and interlocutors through spatiotemporal changes and articulation style. This specifically manifests itself in educational settings, where novel vocabularies are used by teachers, and students. To address this gap, we collect a motion capture dataset of American Sign Language (ASL) STEM (Science, Technology, Engineering, and Mathematics) dialogue that enables quantitative comparison between dyadic interactive signing, solo signed lecture, and interpreted articles. Using continuous kinematic features, we disentangle dialogue-specific entrainment from individual effort reduction and show spatiotemporal changes across repeated mentions of STEM terms. On average, dialogue signs are 24.6%-44.6% shorter in duration than the isolated signs, and show significant reductions absent in monologue contexts. Finally, we evaluate sign embedding models on their ability to recognize STEM signs and approximate how entrained the participants become over time. Our study bridges linguistic analysis and computational modeling to understand how pragmatics shape sign articulation and its representation in sign language technologies.