SinIR: Efficient General Image Manipulation with Single Image Reconstruction

TL;DR

SinIR is a fast, single-image trained framework for diverse image manipulations, achieving competitive results with simpler, more efficient training compared to GAN-based methods.

Contribution

Introduces SinIR, a reconstruction-based single-image framework for multiple image manipulation tasks with significantly faster training and competitive performance.

Findings

33.5 times faster training than SinGAN

Competitive performance on various tasks

Effective control via pixel shuffling

Abstract

We propose SinIR, an efficient reconstruction-based framework trained on a single natural image for general image manipulation, including super-resolution, editing, harmonization, paint-to-image, photo-realistic style transfer, and artistic style transfer. We train our model on a single image with cascaded multi-scale learning, where each network at each scale is responsible for image reconstruction. This reconstruction objective greatly reduces the complexity and running time of training, compared to the GAN objective. However, the reconstruction objective also exacerbates the output quality. Therefore, to solve this problem, we further utilize simple random pixel shuffling, which also gives control over manipulation, inspired by the Denoising Autoencoder. With quantitative evaluation, we show that SinIR has competitive performance on various image manipulation tasks. Moreover, with a much simpler training objective (i.e., reconstruction), SinIR is trained 33.5 times faster than SinGAN (for 500 X 500 images) that solves similar tasks. Our code is publicly available at github.com/YooJiHyeong/SinIR.