SLIM: Semantic-based Low-bitrate Image compression for Machines by leveraging diffusion

TL;DR

SLIM introduces a semantic-based image compression framework optimized for machine vision, focusing on regions of interest and leveraging diffusion models to achieve low bitrate and high classification accuracy.

Contribution

SLIM presents a novel training framework that compresses only RoI areas using a pretrained diffusion model, improving machine vision performance at low bitrates.

Findings

Higher classification accuracy at same bits per pixel compared to traditional methods

Effective RoI-focused compression without guide masks at inference

Enhanced image reconstruction for both machine and human vision

Abstract

In recent years, the demand of image compression models for machine vision has increased dramatically. However, the training frameworks of image compression still focus on the vision of human, maintaining the excessive perceptual details, thus have limitations in optimally reducing the bits per pixel in the case of performing machine vision tasks. In this paper, we propose Semantic-based Low-bitrate Image compression for Machines by leveraging diffusion, termed SLIM. This is a new effective training framework of image compression for machine vision, using a pretrained latent diffusion model.The compressor model of our method focuses only on the Region-of-Interest (RoI) areas for machine vision in the image latent, to compress it compactly. Then the pretrained Unet model enhances the decompressed latent, utilizing a RoI-focused text caption which containing semantic information of the image. Therefore, SLIM is able to focus on RoI areas of the image without any guide mask at the inference stage, achieving low bitrate when compressing. And SLIM is also able to enhance a decompressed latent by denoising steps, so the final reconstructed image from the enhanced latent can be optimized for the machine vision task while still containing perceptual details for human vision. Experimental results show that SLIM achieves a higher classification accuracy in the same bits per pixel condition, compared to conventional image compression models for machines.