Measuring Diversity in Co-creative Image Generation

TL;DR

This paper introduces an entropy-based method for measuring diversity in co-creative image generation that does not rely on ground-truth data and is computationally efficient.

Contribution

It proposes a novel diversity measure using neural network encoding entropy, addressing limitations of existing methods that depend on ground-truth comparisons.

Findings

Entropy-based diversity measure correlates with perceived diversity

Comparison of pre-trained networks influences diversity assessment

Method is computationally efficient and applicable to creative systems

Abstract

Quality and diversity have been proposed as reasonable heuristics for assessing content generated by co-creative systems, but to date there has been little agreement around what constitutes the latter or how to measure it. Proposed approaches for assessing generative models in terms of diversity have limitations in that they compare the model's outputs to a ground truth that in the era of large pre-trained generative models might not be available, or entail an impractical number of computations. We propose an alternative based on entropy of neural network encodings for comparing diversity between sets of images that does not require ground-truth knowledge and is easy to compute. We also compare two pre-trained networks and show how the choice relates to the notion of diversity that we want to evaluate. We conclude with a discussion of the potential applications of these measures for ideation in interactive systems, model evaluation, and more broadly within computational creativity.