Exploring Speech Cues in Web-mined COVID-19 Conversational Vlogs

TL;DR

This study analyzes YouTube COVID-19 vlogs from New York City to identify emotional and linguistic patterns related to pandemic events, revealing acoustic features that reflect emotional reactivity and align with key COVID-19 milestones.

Contribution

It introduces an analysis of acoustic and linguistic cues in social media videos to track emotional responses during the pandemic, highlighting their association with COVID-19 events.

Findings

Acoustic features like jitter and shimmer indicate emotional reactivity.

Patterns in vlog content align with COVID-19 case peaks and emergency declarations.

Some linguistic features also correlate with pandemic milestones.

Abstract

The COVID-19 pandemic caused by the novel SARS-Coronavirus-2 (n-SARS-CoV-2) has impacted people's lives in unprecedented ways. During the time of the pandemic, social vloggers have used social media to actively share their opinions or experiences in quarantine. This paper collected videos from YouTube to track emotional responses in conversational vlogs and their potential associations with events related to the pandemic. In particular, vlogs uploaded from locations in New York City were analyzed given that this was one of the first epicenters of the pandemic in the United States. We observed some common patterns in vloggers' acoustic and linguistic features across the time span of the quarantine, which is indicative of changes in emotional reactivity. Additionally, we investigated fluctuations of acoustic and linguistic patterns in relation to COVID-19 events in the New York area (e.g. the number of daily new cases, number of deaths, and extension of stay-at-home order and state of emergency). Our results indicate that acoustic features, such as zero-crossing-rate, jitter, and shimmer, can be valuable for analyzing emotional reactivity in social media videos. Our findings further indicate that some of the peaks of the acoustic and linguistic indices align with COVID-19 events, such as the peak in the number of deaths and emergency declaration.