Towards creativity characterization of generative models via group-based subset scanning

TL;DR

This paper introduces a group-based subset scanning method to identify and analyze creative outputs in generative models by detecting anomalous activation patterns, enhancing understanding of machine creativity.

Contribution

It proposes a novel subset scanning technique to characterize creativity in generative models through anomaly detection in hidden layer activations.

Findings

Creative samples show larger anomalous subsets than normal samples.

Subset scores are more effective in detecting creativity in activation space.

Creative decoding activates different nodes compared to normal decoding.

Abstract

Deep generative models, such as Variational Autoencoders (VAEs), have been employed widely in computational creativity research. However, such models discourage out-of-distribution generation to avoid spurious sample generation, limiting their creativity. Thus, incorporating research on human creativity into generative deep learning techniques presents an opportunity to make their outputs more compelling and human-like. As we see the emergence of generative models directed to creativity research, a need for machine learning-based surrogate metrics to characterize creative output from these models is imperative. We propose group-based subset scanning to quantify, detect, and characterize creative processes by detecting a subset of anomalous node-activations in the hidden layers of generative models. Our experiments on original, typically decoded, and "creatively decoded" (Das et al 2020) image datasets reveal that the proposed subset scores distribution is more useful for detecting creative processes in the activation space rather than the pixel space. Further, we found that creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets. The node activations highlighted during the creative decoding process are different from those responsible for normal sample generation.