Global Image Sentiment Transfer

TL;DR

This paper introduces a novel framework for transferring the sentiment of images by combining content-related reference image retrieval and an optimization-based sentiment transfer process, improving the quality and reliability of sentiment transfer results.

Contribution

It proposes a new image retrieval algorithm based on SSIM for better content relevance and an optimization-based sentiment transfer method that preserves image structure.

Findings

Outperforms existing style transfer algorithms in sentiment transfer quality.

Uses SSIM-based retrieval for more content-related reference images.

Employs an iterative optimization algorithm for effective sentiment transfer.

Abstract

Transferring the sentiment of an image is an unexplored research topic in the area of computer vision. This work proposes a novel framework consisting of a reference image retrieval step and a global sentiment transfer step to transfer sentiments of images according to a given sentiment tag. The proposed image retrieval algorithm is based on the SSIM index. The retrieved reference images by the proposed algorithm are more content-related against the algorithm based on the perceptual loss. Therefore can lead to a better image sentiment transfer result. In addition, we propose a global sentiment transfer step, which employs an optimization algorithm to iteratively transfer sentiment of images based on feature maps produced by the Densenet121 architecture. The proposed sentiment transfer algorithm can transfer the sentiment of images while ensuring the content structure of the input image intact. The qualitative and quantitative experiments demonstrate that the proposed sentiment transfer framework outperforms existing artistic and photorealistic style transfer algorithms in making reliable sentiment transfer results with rich and exact details.