An Exploration of Active Learning for Affective Digital Phenotyping

TL;DR

This paper investigates active learning strategies to improve data labeling efficiency in affective computing, demonstrating slight performance gains in emotion recognition from complex, subjective datasets.

Contribution

It introduces two novel active learning methods tailored for subjective affective data, leveraging gameplay metadata and crowdsourced label entropy.

Findings

Active learning with gameplay metadata slightly outperforms random selection.

Prioritizing frames by crowdsourced label entropy reduces cross-entropy loss.

Pilot evaluations show potential for active learning in noisy, subjective affective datasets.

Abstract

Some of the most severe bottlenecks preventing widespread development of machine learning models for human behavior include a dearth of labeled training data and difficulty of acquiring high quality labels. Active learning is a paradigm for using algorithms to computationally select a useful subset of data points to label using metrics for model uncertainty and data similarity. We explore active learning for naturalistic computer vision emotion data, a particularly heterogeneous and complex data space due to inherently subjective labels. Using frames collected from gameplay acquired from a therapeutic smartphone game for children with autism, we run a simulation of active learning using gameplay prompts as metadata to aid in the active learning process. We find that active learning using information generated during gameplay slightly outperforms random selection of the same number of labeled frames. We next investigate a method to conduct active learning with subjective data, such as in affective computing, and where multiple crowdsourced labels can be acquired for each image. Using the Child Affective Facial Expression (CAFE) dataset, we simulate an active learning process for crowdsourcing many labels and find that prioritizing frames using the entropy of the crowdsourced label distribution results in lower categorical cross-entropy loss compared to random frame selection. Collectively, these results demonstrate pilot evaluations of two novel active learning approaches for subjective affective data collected in noisy settings.