Perspectives on Capturing Emotional Expressiveness in Sign Language

TL;DR

This paper explores how emotions are expressed and perceived in sign language through interviews, highlighting manual and non-manual cues, and discusses challenges and design considerations for creating emotionally-aware sign language translation technologies.

Contribution

It provides new insights into emotional expression in sign language and offers practical design considerations for developing emotionally-aware sign language technologies.

Findings

Manual and non-manual cues are crucial for emotional expression in sign language.

Universal patterns and cultural variations influence how emotions are communicated.

Challenges exist in capturing emotional nuance for sign language translation.

Abstract

Significant advances have been made in our ability to understand and generate emotionally expressive content such as text and speech, yet comparable progress in sign language technologies remain limited. While computational approaches to sign language translation have focused on capturing lexical content, the emotional dimensions of sign language communication remain largely unexplored. Through semi-structured interviews with eight sign language users across Singapore, Sri Lanka and the United States, including both Deaf and Hard of hearing (DHH) and hearing signers, we investigate how emotions are expressed and perceived in sign languages. Our findings highlight the role of both manual and non-manual elements in emotional expression, revealing universal patterns as well as individual and cultural variations in how signers communicate emotions. We identify key challenges in capturing emotional nuance for sign language translation, and propose design considerations for developing more emotionally-aware sign language technologies. This work contributes to both theoretical understanding of emotional expression in sign language and practical development of interfaces to better serve diverse signing communities.