Automatic Creativity Measurement in Scratch Programs Across Modalities

TL;DR

This paper develops a formal, computable measure of creativity based on core concepts like fluency, flexibility, and originality, and applies it to Scratch programming projects using machine learning trained on expert assessments.

Contribution

It introduces a general, theoretically grounded creativity measure adapted for Scratch and demonstrates its effectiveness through a machine learning model trained on human expert evaluations.

Findings

Automatic assessments aligned better with some experts than experts did with each other.

The measure captures variability in expert opinions on creativity.

First application of computational creativity measurement in educational programming.

Abstract

Promoting creativity is considered an important goal of education, but creativity is notoriously hard to measure.In this paper, we make the journey fromdefining a formal measure of creativity that is efficientlycomputable to applying the measure in a practical domain. The measure is general and relies on coretheoretical concepts in creativity theory, namely fluency, flexibility, and originality, integratingwith prior cognitive science literature. We adapted the general measure for projects in the popular visual programming language Scratch.We designed a machine learning model for predicting the creativity of Scratch projects, trained and evaluated on human expert creativity assessments in an extensive user study. Our results show that opinions about creativity in Scratch varied widely across experts. The automatic creativity assessment aligned with the assessment of the human experts more than the experts agreed with each other. This is a first step in providing computational models for measuring creativity that can be applied to educational technologies, and to scale up the benefit of creativity education in schools.