MECO: A Multimodal Dataset for Emotion and Cognitive Understanding in Older Adults

TL;DR

MECO is a comprehensive multimodal dataset designed to advance emotion and cognitive understanding in older adults, addressing a critical gap in affective computing research for aging populations.

Contribution

The paper introduces MECO, a novel dataset with multimodal signals and annotations for emotion and cognition in older adults, enabling new research in this underexplored area.

Findings

Established baseline benchmarks for emotion and cognitive prediction.

Collected 38 hours of multimodal data from 42 older adults.

Provided the dataset publicly at https://maitrechen.github.io/meco-page/.

Abstract

While affective computing has advanced considerably, multimodal emotion prediction in aging populations remains underexplored, largely due to the scarcity of dedicated datasets. Existing multimodal benchmarks predominantly target young, cognitively healthy subjects, neglecting the influence of cognitive decline on emotional expression and physiological responses. To bridge this gap, we present MECO, a Multimodal dataset for Emotion and Cognitive understanding in Older adults. MECO includes 42 participants and provides approximately 38 hours of multimodal signals, yielding 30,592 synchronized samples. To maximize ecological validity, data collection followed standardized protocols within community-based settings. The modalities cover video, audio, electroencephalography (EEG), and electrocardiography (ECG). In addition, the dataset offers comprehensive annotations of emotional and cognitive states, including self-assessed valence, arousal, six basic emotions, and Mini-Mental State Examination cognitive scores. We further establish baseline benchmarks for both emotion and cognitive prediction. MECO serves as a foundational resource for multimodal modeling of affect and cognition in aging populations, facilitating downstream applications such as personalized emotion recognition and early detection of mild cognitive impairment (MCI) in real-world settings. The complete dataset and supplementary materials are available at https://maitrechen.github.io/meco-page/.