A Novel Framework for Image-to-image Translation and Image Compression

TL;DR

This paper introduces a unified neural framework combining image-to-image translation and image compression, enabling flexible multi-domain image synthesis and autoencoding with promising experimental results.

Contribution

It presents a novel joint I2I translation and compression framework with adaptive mode conditioning, unifying translation and autoencoding in a single model.

Findings

Effective multi-domain image synthesis and compression

Single model achieves promising results in both tasks

Flexible adaptation via mode-conditioned residual blocks

Abstract

Data-driven paradigms using machine learning are becoming ubiquitous in image processing and communications. In particular, image-to-image (I2I) translation is a generic and widely used approach to image processing problems, such as image synthesis, style transfer, and image restoration. At the same time, neural image compression has emerged as a data-driven alternative to traditional coding approaches in visual communications. In this paper, we study the combination of these two paradigms into a joint I2I compression and translation framework, focusing on multi-domain image synthesis. We first propose distributed I2I translation by integrating quantization and entropy coding into an I2I translation framework (i.e. I2Icodec). In practice, the image compression functionality (i.e. autoencoding) is also desirable, requiring to deploy alongside I2Icodec a regular image codec. Thus, we further propose a unified framework that allows both translation and autoencoding capabilities in a single codec. Adaptive residual blocks conditioned on the translation/compression mode provide flexible adaptation to the desired functionality. The experiments show promising results in both I2I translation and image compression using a single model.