A Taxonomy of Miscompressions: Preparing Image Forensics for Neural Compression

TL;DR

This paper introduces a taxonomy for categorizing miscompressions in neural image compression, focusing on semantic fidelity issues and their impact, to improve detection and mitigation strategies.

Contribution

It proposes a novel taxonomy of miscompressions in neural image compression, aiding understanding and research into their detection and mitigation.

Findings

Identifies three types of miscompression effects.

Defines a binary flag for high-impact miscompressions.

Facilitates risk communication and mitigation research.

Abstract

Neural compression has the potential to revolutionize lossy image compression. Based on generative models, recent schemes achieve unprecedented compression rates at high perceptual quality but compromise semantic fidelity. Details of decompressed images may appear optically flawless but semantically different from the originals, making compression errors difficult or impossible to detect. We explore the problem space and propose a provisional taxonomy of miscompressions. It defines three types of 'what happens' and has a binary 'high impact' flag indicating miscompressions that alter symbols. We discuss how the taxonomy can facilitate risk communication and research into mitigations.