"That's so cute!": The CARE Dataset for Affective Response Detection

TL;DR

This paper introduces CARE_db, a large social media dataset annotated with affective responses using a novel comment-based method, enabling effective training of models for affect detection without costly human annotation.

Contribution

The paper presents a scalable, comment-based annotation method for affective responses and a large dataset, improving affect detection without extensive human labeling.

Findings

CARE_db outperforms crowd-sourced annotations in accuracy.

BERT-based models trained on CARE_db effectively predict affective responses.

The method enables scalable annotation for new affective responses.

Abstract

Social media plays an increasing role in our communication with friends and family, and our consumption of information and entertainment. Hence, to design effective ranking functions for posts on social media, it would be useful to predict the affective response to a post (e.g., whether the user is likely to be humored, inspired, angered, informed). Similar to work on emotion recognition (which focuses on the affect of the publisher of the post), the traditional approach to recognizing affective response would involve an expensive investment in human annotation of training data. We introduce CARE$_{db}$, a dataset of 230k social media posts annotated according to 7 affective responses using the Common Affective Response Expression (CARE) method. The CARE method is a means of leveraging the signal that is present in comments that are posted in response to a post, providing high-precision evidence about the affective response of the readers to the post without human annotation. Unlike human annotation, the annotation process we describe here can be iterated upon to expand the coverage of the method, particularly for new affective responses. We present experiments that demonstrate that the CARE annotations compare favorably with crowd-sourced annotations. Finally, we use CARE$_{db}$ to train competitive BERT-based models for predicting affective response as well as emotion detection, demonstrating the utility of the dataset for related tasks.