CPIPS: Learning to Preserve Perceptual Distances in End-to-End Image Compression

TL;DR

CPIPS is a neural codec-based perceptual distance measure that efficiently preserves semantic relevance and perceptual similarity in compressed images, benefiting machine vision applications.

Contribution

We introduce CPIPS, a novel perceptual distance metric derived from neural codecs that is faster and more efficient than existing DNN-based metrics.

Findings

CPIPS preserves perceptual distances effectively.

CPIPS is significantly faster than LPIPS and DISTS.

CPIPS is derived with minimal additional cost from neural codecs.

Abstract

Lossy image coding standards such as JPEG and MPEG have successfully achieved high compression rates for human consumption of multimedia data. However, with the increasing prevalence of IoT devices, drones, and self-driving cars, machines rather than humans are processing a greater portion of captured visual content. Consequently, it is crucial to pursue an efficient compressed representation that caters not only to human vision but also to image processing and machine vision tasks. Drawing inspiration from the efficient coding hypothesis in biological systems and the modeling of the sensory cortex in neural science, we repurpose the compressed latent representation to prioritize semantic relevance while preserving perceptual distance. Our proposed method, Compressed Perceptual Image Patch Similarity (CPIPS), can be derived at a minimal cost from a learned neural codec and computed significantly faster than DNN-based perceptual metrics such as LPIPS and DISTS.