Quantifying the Academic Quality of Children's Videos using Machine Comprehension

TL;DR

This paper introduces a method to quantify the academic quality of children's videos on YouTube Kids by using a reading comprehension model to assess how well videos teach school-relevant content, providing a new metric for educational value.

Contribution

The study develops a novel approach to measure educational quality of children's videos using RC models and applies it to analyze top channels on YouTube Kids, addressing the challenge of defining learning.

Findings

RC model effectively estimates academic learning from videos.

Top channels vary significantly in educational quality.

Over 80,000 videos analyzed for academic content.

Abstract

YouTube Kids (YTK) is one of the most popular kids' applications used by millions of kids daily. However, various studies have highlighted concerns about the videos on the platform, like the over-presence of entertaining and commercial content. YouTube recently proposed high-quality guidelines that include `promoting learning' and proposed to use it in ranking channels. However, the concept of learning is multi-faceted, and it can be difficult to define and measure in the context of online videos. This research focuses on learning in terms of what's taught in schools and proposes a way to measure the academic quality of children's videos. Using a new dataset of questions and answers from children's videos, we first show that a Reading Comprehension (RC) model can estimate academic learning. Then, using a large dataset of middle school textbook questions on diverse topics, we quantify the academic quality of top channels as the number of children's textbook questions that an RC model can correctly answer. By analyzing over 80,000 videos posted on the top 100 channels, we present the first thorough analysis of the academic quality of channels on YTK.