Disentangle Perceptual Learning through Online Contrastive Learning

TL;DR

This paper introduces an online contrastive learning method to disentangle perception-relevant features from irrelevant ones in image transformation tasks, improving perceptual quality beyond traditional pre-trained network approaches.

Contribution

It proposes a novel online contrastive learning framework that selectively activates perception-relevant features, enhancing image transformation quality based on human visual perception.

Findings

Outperforms existing methods in perceptual quality

Effectively disentangles perception-relevant features

Improves image transformation results

Abstract

Pursuing realistic results according to human visual perception is the central concern in the image transformation tasks. Perceptual learning approaches like perceptual loss are empirically powerful for such tasks but they usually rely on the pre-trained classification network to provide features, which are not necessarily optimal in terms of visual perception of image transformation. In this paper, we argue that, among the features representation from the pre-trained classification network, only limited dimensions are related to human visual perception, while others are irrelevant, although both will affect the final image transformation results. Under such an assumption, we try to disentangle the perception-relevant dimensions from the representation through our proposed online contrastive learning. The resulted network includes the pre-training part and a feature selection layer, followed by the contrastive learning module, which utilizes the transformed results, target images, and task-oriented distorted images as the positive, negative, and anchor samples, respectively. The contrastive learning aims at activating the perception-relevant dimensions and suppressing the irrelevant ones by using the triplet loss, so that the original representation can be disentangled for better perceptual quality. Experiments on various image transformation tasks demonstrate the superiority of our framework, in terms of human visual perception, to the existing approaches using pre-trained networks and empirically designed losses.