Real-time 6K Image Rescaling with Rate-distortion Optimization

TL;DR

This paper introduces HyperThumbnail, a real-time framework for 6K image rescaling that optimizes file size and reconstruction quality by embedding high-resolution images into JPEG thumbnails with a novel encoder and frequency-aware decoder.

Contribution

The paper presents a novel rate-distortion-aware image rescaling framework that enables real-time 6K image reconstruction with optimized embedding and decoding techniques.

Findings

Outperforms previous methods in rate-distortion performance.

Achieves real-time 6K image reconstruction.

Efficiently embeds high-resolution images into JPEG thumbnails.

Abstract

Contemporary image rescaling aims at embedding a high-resolution (HR) image into a low-resolution (LR) thumbnail image that contains embedded information for HR image reconstruction. Unlike traditional image super-resolution, this enables high-fidelity HR image restoration faithful to the original one, given the embedded information in the LR thumbnail. However, state-of-the-art image rescaling methods do not optimize the LR image file size for efficient sharing and fall short of real-time performance for ultra-high-resolution (e.g., 6K) image reconstruction. To address these two challenges, we propose a novel framework (HyperThumbnail) for real-time 6K rate-distortion-aware image rescaling. Our framework first embeds an HR image into a JPEG LR thumbnail by an encoder with our proposed quantization prediction module, which minimizes the file size of the embedding LR JPEG thumbnail while maximizing HR reconstruction quality. Then, an efficient frequency-aware decoder reconstructs a high-fidelity HR image from the LR one in real time. Extensive experiments demonstrate that our framework outperforms previous image rescaling baselines in rate-distortion performance and can perform 6K image reconstruction in real time.