JPEG AI Image Compression Visual Artifacts: Detection Methods and Dataset

TL;DR

This paper introduces methods to detect, localize, and quantify visual artifacts caused by neural network-based image compression, and provides a large validated dataset for testing and improving such codecs.

Contribution

It presents novel artifact detection and localization techniques, along with a comprehensive dataset of artifacts from neural compression methods, aiding future research and development.

Findings

Detected and localized three types of artifacts in neural compression images.

Created a dataset of 46,440 validated artifact examples from 350,000 images.

Provided publicly available source code and dataset for community use.

Abstract

Learning-based image compression methods have improved in recent years and started to outperform traditional codecs. However, neural-network approaches can unexpectedly introduce visual artifacts in some images. We therefore propose methods to separately detect three types of artifacts (texture and boundary degradation, color change, and text corruption), to localize the affected regions, and to quantify the artifact strength. We consider only those regions that exhibit distortion due solely to the neural compression but that a traditional codec recovers successfully at a comparable bitrate. We employed our methods to collect artifacts for the JPEG AI verification model with respect to HM-18.0, the H.265 reference software. We processed about 350,000 unique images from the Open Images dataset using different compression-quality parameters; the result is a dataset of 46,440 artifacts validated through crowd-sourced subjective assessment. Our proposed dataset and methods are valuable for testing neural-network-based image codecs, identifying bugs in these codecs, and enhancing their performance. We make source code of the methods and the dataset publicly available.