Emotion recognition using a glasses-type wearable device via multi-channel facial responses

TL;DR

This paper introduces a glasses-type wearable device that unobtrusively captures facial physiological signals and images to accurately recognize human emotions, demonstrating improved accuracy with multi-channel data collection.

Contribution

The study develops a novel wearable device that combines physiological sensors and a camera for emotion recognition, with validated algorithms showing enhanced accuracy over single-channel methods.

Findings

Emotion recognition accuracy reached 78% using facial expressions alone.

Multi-channel data increased recognition accuracy by 10.1%.

The device is suitable for daily life emotion monitoring in healthcare.

Abstract

We present a glasses type wearable device to detect emotions from a human face in an unobtrusive manner. The device is designed to gather multi channel responses from the user face naturally and continuously while the user is wearing it. The multi channel responses include physiological responses of the facial muscles and organs based on electrodermal activity (EDA) and photoplethysmogram. We conducted experiments to determine the optimal positions of EDA sensors on the wearable device because EDA signal quality is very sensitive to the sensing position. In addition to the physiological data, the device can capture the image region representing local facial expressions around the left eye via a built in camera. In this study, we developed and validated an algorithm to recognize emotions using multi channel responses obtained from the device. The results show that the emotion recognition algorithm using only local facial expressions has an accuracy of 78 percent at classifying emotions. Using multi channel data, this accuracy was increased by 10.1 percent. This unobtrusive wearable system with facial multi channel responses is very useful for monitoring a user emotions in daily life, which has a huge potential for use in the healthcare industry.