CodeSCAN: ScreenCast ANalysis for Video Programming Tutorials

TL;DR

This paper introduces CodeSCAN, a large dataset of 12,000 screenshots from programming tutorials, and benchmarks for analyzing IDE elements, color conversion, and OCR to improve video programming tutorial analysis.

Contribution

The paper presents the first large-scale, diverse dataset for screencast analysis and provides benchmarks for key tasks like IDE element detection and OCR.

Findings

Benchmark results for IDE element detection, color conversion, and OCR.

Diverse dataset enables better analysis of programming screencasts.

Facilitates future research in coding screencast understanding.

Abstract

Programming tutorials in the form of coding screencasts play a crucial role in programming education, serving both novices and experienced developers. However, the video format of these tutorials presents a challenge due to the difficulty of searching for and within videos. Addressing the absence of large-scale and diverse datasets for screencast analysis, we introduce the CodeSCAN dataset. It comprises 12,000 screenshots captured from the Visual Studio Code environment during development, featuring 24 programming languages, 25 fonts, and over 90 distinct themes, in addition to diverse layout changes and realistic user interactions. Moreover, we conduct detailed quantitative and qualitative evaluations to benchmark the performance of Integrated Development Environment (IDE) element detection, color-to-black-and-white conversion, and Optical Character Recognition (OCR). We hope that our contributions facilitate more research in coding screencast analysis, and we make the source code for creating the dataset and the benchmark publicly available on this website.