Assessing a Single Image in Reference-Guided Image Synthesis

TL;DR

This paper introduces RISA, a learning-based framework for evaluating the quality of individual images generated in reference-guided image synthesis, without requiring human annotations, and demonstrates its effectiveness and consistency with human preferences.

Contribution

The paper proposes RISA, a novel, annotation-free evaluation method for single image quality assessment in RIS tasks, using weak supervision, label refinement, and contrastive learning.

Findings

RISA correlates highly with human preferences.

It transfers well across different models.

It outperforms existing metrics in RIS evaluation.

Abstract

Assessing the performance of Generative Adversarial Networks (GANs) has been an important topic due to its practical significance. Although several evaluation metrics have been proposed, they generally assess the quality of the whole generated image distribution. For Reference-guided Image Synthesis (RIS) tasks, i.e., rendering a source image in the style of another reference image, where assessing the quality of a single generated image is crucial, these metrics are not applicable. In this paper, we propose a general learning-based framework, Reference-guided Image Synthesis Assessment (RISA) to quantitatively evaluate the quality of a single generated image. Notably, the training of RISA does not require human annotations. In specific, the training data for RISA are acquired by the intermediate models from the training procedure in RIS, and weakly annotated by the number of models' iterations, based on the positive correlation between image quality and iterations. As this annotation is too coarse as a supervision signal, we introduce two techniques: 1) a pixel-wise interpolation scheme to refine the coarse labels, and 2) multiple binary classifiers to replace a na\"ive regressor. In addition, an unsupervised contrastive loss is introduced to effectively capture the style similarity between a generated image and its reference image. Empirical results on various datasets demonstrate that RISA is highly consistent with human preference and transfers well across models.