Random Network Distillation as a Diversity Metric for Both Image and Text Generation

TL;DR

This paper introduces a novel diversity metric based on random network distillation, applicable to various data types like images and text, and demonstrates its effectiveness in evaluating generative models and few-shot image generation.

Contribution

The paper presents a new diversity metric using random network distillation that works across data types and improves evaluation of generative models and few-shot learning.

Findings

Effective in quantifying data diversity for images and text

Useful in evaluating few-shot image generation

Outperforms existing diversity metrics

Abstract

Generative models are increasingly able to produce remarkably high quality images and text. The community has developed numerous evaluation metrics for comparing generative models. However, these metrics do not effectively quantify data diversity. We develop a new diversity metric that can readily be applied to data, both synthetic and natural, of any type. Our method employs random network distillation, a technique introduced in reinforcement learning. We validate and deploy this metric on both images and text. We further explore diversity in few-shot image generation, a setting which was previously difficult to evaluate.