Towards improved lossy image compression: Human image reconstruction with public-domain images

TL;DR

This paper introduces a human-centric paradigm for lossy image compression where humans describe and reconstruct images, leveraging public datasets, leading to improved semantic preservation over traditional methods like WebP.

Contribution

The paper proposes a novel human-based image reconstruction paradigm that utilizes public images and text instructions to enhance lossy compression quality.

Findings

Human reconstructions outperform WebP at low bitrates.

Prioritizing semantic elements improves compression results.

The paradigm enables development of human-centric loss functions.

Abstract

Lossy image compression has been studied extensively in the context of typical loss functions such as RMSE, MS-SSIM, etc. However, compression at low bitrates generally produces unsatisfying results. Furthermore, the availability of massive public image datasets appears to have hardly been exploited in image compression. Here, we present a paradigm for eliciting human image reconstruction in order to perform lossy image compression. In this paradigm, one human describes images to a second human, whose task is to reconstruct the target image using publicly available images and text instructions. The resulting reconstructions are then evaluated by human raters on the Amazon Mechanical Turk platform and compared to reconstructions obtained using state-of-the-art compressor WebP. Our results suggest that prioritizing semantic visual elements may be key to achieving significant improvements in image compression, and that our paradigm can be used to develop a more human-centric loss function. The images, results and additional data are available at https://compression.stanford.edu/human-compression