CGI: Identifying Conditional Generative Models with Example Images

TL;DR

This paper introduces CGI, a method for identifying suitable generative models using user-provided images, improving search efficiency in large model hubs, and demonstrating high accuracy and better FID scores.

Contribution

It proposes the PMI approach for precise model description and matching, along with a benchmark of 65 models and 9100 tasks to evaluate effectiveness.

Findings

92% model identification accuracy with four example images

Significantly improved FID scores

Effective human evaluation results

Abstract

Generative models have achieved remarkable performance recently, and thus model hubs have emerged. Existing model hubs typically assume basic text matching is sufficient to search for models. However, in reality, due to different abstractions and the large number of models in model hubs, it is not easy for users to review model descriptions and example images, choosing which model best meets their needs. Therefore, it is necessary to describe model functionality wisely so that future users can efficiently search for the most suitable model for their needs. Efforts to address this issue remain limited. In this paper, we propose Conditional Generative Model Identification (CGI), which aims to provide an effective way to identify the most suitable model using user-provided example images rather than requiring users to manually review a large number of models with example images. To address this problem, we propose the PromptBased Model Identification (PMI) , which can adequately describe model functionality and precisely match requirements with specifications. To evaluate PMI approach and promote related research, we provide a benchmark comprising 65 models and 9100 identification tasks. Extensive experimental and human evaluation results demonstrate that PMI is effective. For instance, 92% of models are correctly identified with significantly better FID scores when four example images are provided.