Towards Bidirectional Arbitrary Image Rescaling: Joint Optimization and Cycle Idempotence

TL;DR

This paper introduces a unified bidirectional image rescaling model that jointly optimizes for arbitrary upscaling and downscaling, achieving high performance and robustness in cycle consistency, applicable to real-world scenarios.

Contribution

It is the first model to treat arbitrary image rescaling as a unified process, enabling simultaneous learning of upscaling and downscaling with cycle idempotence.

Findings

Significantly outperforms existing arbitrary upscaling models.

Maintains high visual quality in downscaled images.

Robust to repeated cycle applications and large/asymmetric scales.

Abstract

Deep learning based single image super-resolution models have been widely studied and superb results are achieved in upscaling low-resolution images with fixed scale factor and downscaling degradation kernel. To improve real world applicability of such models, there are growing interests to develop models optimized for arbitrary upscaling factors. Our proposed method is the first to treat arbitrary rescaling, both upscaling and downscaling, as one unified process. Using joint optimization of both directions, the proposed model is able to learn upscaling and downscaling simultaneously and achieve bidirectional arbitrary image rescaling. It improves the performance of current arbitrary upscaling models by a large margin while at the same time learns to maintain visual perception quality in downscaled images. The proposed model is further shown to be robust in cycle idempotence test, free of severe degradations in reconstruction accuracy when the downscaling-to-upscaling cycle is applied repetitively. This robustness is beneficial for image rescaling in the wild when this cycle could be applied to one image for multiple times. It also performs well on tests with arbitrary large scales and asymmetric scales, even when the model is not trained with such tasks. Extensive experiments are conducted to demonstrate the superior performance of our model.