VVC+M: Plug and Play Scalable Image Coding for Humans and Machines

TL;DR

This paper introduces VVC+M, a scalable image coding method that leverages existing video codec residual coding to optimize machine analysis performance without sacrificing human perception quality.

Contribution

It proposes a post-hoc, adaptable scalable coding scheme that enhances machine task performance while maintaining competitive human visual quality, using residual coding capabilities of VVC.

Findings

Improves rate-distortion performance for machine analysis tasks.

Maintains competitive quality for human perception.

Can be applied post-hoc to existing ICM schemes.

Abstract

Compression for machines is an emerging field, where inputs are encoded while optimizing the performance of downstream automated analysis. In scalable coding for humans and machines, the compressed representation used for machines is further utilized to enable input reconstruction. Often performed by jointly optimizing the compression scheme for both machine task and human perception, this results in sub-optimal rate-distortion (RD) performance for the machine side. We focus on the case of images, proposing to utilize the pre-existing residual coding capabilities of video codecs such as VVC to create a scalable codec from any image compression for machines (ICM) scheme. Using our approach we improve an existing scalable codec to achieve superior RD performance on the machine task, while remaining competitive for human perception. Moreover, our approach can be trained post-hoc for any given ICM scheme, and without creating a coupling between the quality of the machine analysis and human vision.