A Multi-Armed Bandit Approach to Online Selection and Evaluation of Generative Models

TL;DR

This paper introduces an online, sample-efficient framework for evaluating and selecting the best generative models using multi-armed bandit algorithms, specifically UCB variants, to optimize evaluation metrics like Fréchet Distance and Inception Score.

Contribution

It proposes a novel online evaluation method for generative models using MAB algorithms, with theoretical regret bounds and empirical validation on image datasets.

Findings

MAB algorithms effectively identify high-quality generative models with fewer samples.

The proposed FD-UCB and IS-UCB algorithms outperform baseline methods in sample efficiency.

Empirical results demonstrate the approach's potential for practical model selection in generative modeling.

Abstract

Existing frameworks for evaluating and comparing generative models consider an offline setting, where the evaluator has access to large batches of data produced by the models. However, in practical scenarios, the goal is often to identify and select the best model using the fewest possible generated samples to minimize the costs of querying data from the sub-optimal models. In this work, we propose an online evaluation and selection framework to find the generative model that maximizes a standard assessment score among a group of available models. We view the task as a multi-armed bandit (MAB) and propose upper confidence bound (UCB) bandit algorithms to identify the model producing data with the best evaluation score that quantifies the quality and diversity of generated data. Specifically, we develop the MAB-based selection of generative models considering the Fr\'echet Distance (FD) and Inception Score (IS) metrics, resulting in the FD-UCB and IS-UCB algorithms. We prove regret bounds for these algorithms and present numerical results on standard image datasets. Our empirical results suggest the efficacy of MAB approaches for the sample-efficient evaluation and selection of deep generative models. The project code is available at https://github.com/yannxiaoyanhu/dgm-online-eval.