The Role of Facial Expressions and Emotion in ASL

TL;DR

This paper investigates the relationship between facial expressions and emotional content in American Sign Language, demonstrating that facial cues can predict emotional categories and contribute to understanding sign language communication.

Contribution

It introduces two methods for analyzing facial expressions and emotion in ASL, showing that facial features can be used to predict emotional states and interpret signs.

Findings

Facial expressions are significantly related to emotionality in ASL.

A simple classifier can predict emotional categories from facial features.

Many relationships exist between facial cues and emotional content in sign language.

Abstract

There is little prior work on quantifying the relationships between facial expressions and emotionality in American Sign Language. In this final report, we provide two methods for studying these relationships through probability and prediction. Using a large corpus of natural signing manually annotated with facial features paired with lexical emotion datasets, we find that there exist many relationships between emotionality and the face, and that a simple classifier can predict what someone is saying in terms of broad emotional categories only by looking at the face.