Contrastive Feature Loss for Image Prediction

TL;DR

This paper introduces a contrastive feature loss based on mutual information to improve the perceptual quality of image predictions, reducing blurriness and artifacts without relying solely on GANs.

Contribution

It proposes a novel, information theory-based contrastive loss that calibrates feature space for better image reconstruction quality.

Findings

Enhances perceptual realism of generated images

Reduces blurriness and artifacts compared to L1 loss

Works effectively as a drop-in replacement for traditional losses

Abstract

Training supervised image synthesis models requires a critic to compare two images: the ground truth to the result. Yet, this basic functionality remains an open problem. A popular line of approaches uses the L1 (mean absolute error) loss, either in the pixel or the feature space of pretrained deep networks. However, we observe that these losses tend to produce overly blurry and grey images, and other techniques such as GANs need to be employed to fight these artifacts. In this work, we introduce an information theory based approach to measuring similarity between two images. We argue that a good reconstruction should have high mutual information with the ground truth. This view enables learning a lightweight critic to "calibrate" a feature space in a contrastive manner, such that reconstructions of corresponding spatial patches are brought together, while other patches are repulsed. We show that our formulation immediately boosts the perceptual realism of output images when used as a drop-in replacement for the L1 loss, with or without an additional GAN loss.