Beyond the Lens: Quantifying the Impact of Scientific Documentaries through Amazon Reviews

TL;DR

This paper presents a quantitative approach to assess the public impact of scientific documentaries by analyzing Amazon reviews using machine learning, introducing a new impact taxonomy, and releasing a related annotated dataset.

Contribution

It introduces a novel impact category taxonomy, a dataset of annotated reviews, and evaluates machine learning models for impact and sentiment analysis of scientific documentaries.

Findings

Machine learning models can effectively analyze documentary impact.

Annotated dataset enables detailed sentiment and impact analysis.

Scientific documentaries show measurable public engagement.

Abstract

Engaging the public with science is critical for a well-informed population. A popular method of scientific communication is documentaries. Once released, it can be difficult to assess the impact of such works on a large scale, due to the overhead required for in-depth audience feedback studies. In what follows, we overview our complementary approach to qualitative studies through quantitative impact and sentiment analysis of Amazon reviews for several scientific documentaries. In addition to developing a novel impact category taxonomy for this analysis, we release a dataset containing 1296 human-annotated sentences from 1043 Amazon reviews for six movies created in whole or part by the Advanced Visualization Lab (AVL). This interdisciplinary team is housed at the National Center for Supercomputing Applications and consists of visualization designers who focus on cinematic presentations of scientific data. Using this data, we train and evaluate several machine learning and large language models, discussing their effectiveness and possible generalizability for documentaries beyond those focused on for this work. Themes are also extracted from our annotated dataset which, along with our large language model analysis, demonstrate a measure of the ability of scientific documentaries to engage with the public.