Towards Creativity Characterization of Generative Models via Group-based Subset Scanning

TL;DR

This paper introduces a novel group-based subset scanning method to identify and characterize creative outputs in deep generative models by analyzing anomalous node activations, linking model behavior to human perceptions of creativity.

Contribution

It proposes a new technique for detecting and analyzing creative processes in generative models through subset scanning of hidden layer activations, bridging machine metrics with human creativity perception.

Findings

Creative samples show larger anomalous subsets than normal samples.

Subset scores are more effective in activation space for detecting creativity.

Human evaluators agree that images from selected subsets are perceived as more creative.

Abstract

Deep generative models, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), have been employed widely in computational creativity research. However, such models discourage out-of-distribution generation to avoid spurious sample generation, thereby limiting their creativity. Thus, incorporating research on human creativity into generative deep learning techniques presents an opportunity to make their outputs more compelling and human-like. As we see the emergence of generative models directed toward creativity research, a need for machine learning-based surrogate metrics to characterize creative output from these models is imperative. We propose group-based subset scanning to identify, quantify, and characterize creative processes by detecting a subset of anomalous node-activations in the hidden layers of the generative models. Our experiments on the standard image benchmarks, and their "creatively generated" variants, reveal that the proposed subset scores distribution is more useful for detecting creative processes in the activation space rather than the pixel space. Further, we found that creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets. The node activations highlighted during the creative decoding process are different from those responsible for the normal sample generation. Lastly, we assess if the images from the subsets selected by our method were also found creative by human evaluators, presenting a link between creativity perception in humans and node activations within deep neural nets.