Low-Rank Adaptation of Pre-trained Vision Backbones for Energy-Efficient Image Coding for Machine

TL;DR

This paper introduces an energy-efficient image coding framework for machines that uses pre-trained vision models with low-rank adaptation to achieve multi-task versatility and high compression efficiency, reducing energy and storage costs.

Contribution

It proposes a novel low-rank adaptation mechanism for pre-trained vision backbones, enabling multi-task image coding with minimal training and energy consumption.

Findings

Outperforms traditional codecs in efficiency and quality

Reduces training parameters and energy costs

Enables multi-task adaptation without full fine-tuning

Abstract

Image Coding for Machines (ICM) focuses on optimizing image compression for AI-driven analysis rather than human perception. Existing ICM frameworks often rely on separate codecs for specific tasks, leading to significant storage requirements, training overhead, and computational complexity. To address these challenges, we propose an energy-efficient framework that leverages pre-trained vision backbones to extract robust and versatile latent representations suitable for multiple tasks. We introduce a task-specific low-rank adaptation mechanism, which refines the pre-trained features to be both compressible and tailored to downstream applications. This design minimizes trainable parameters and reduces energy costs for multi-task scenarios. By jointly optimizing task performance and entropy minimization, our method enables efficient adaptation to diverse tasks and datasets without full fine-tuning, achieving high coding efficiency. Extensive experiments demonstrate that our framework significantly outperforms traditional codecs and pre-processors, offering an energy-efficient and effective solution for ICM applications. The code and the supplementary materials will be available at: https://gitlab.com/viper-purdue/efficient-compression.