Image Coding for Machines with Omnipotent Feature Learning

TL;DR

This paper introduces Omni-ICM, a novel image coding framework that learns universal, compact features optimized for AI tasks, outperforming traditional codecs by integrating self-supervised learning with an innovative information filtering module.

Contribution

The paper proposes Omni-ICM, a framework that learns omnipotent features for image compression tailored to AI tasks, using a novel information filtering module for better feature generalization.

Findings

Outperforms traditional codecs on multiple vision tasks

Effectively integrates SSL with compression for universal features

Uses a novel information filtering module for adaptive redundancy removal

Abstract

Image Coding for Machines (ICM) aims to compress images for AI tasks analysis rather than meeting human perception. Learning a kind of feature that is both general (for AI tasks) and compact (for compression) is pivotal for its success. In this paper, we attempt to develop an ICM framework by learning universal features while also considering compression. We name such features as omnipotent features and the corresponding framework as Omni-ICM. Considering self-supervised learning (SSL) improves feature generalization, we integrate it with the compression task into the Omni-ICM framework to learn omnipotent features. However, it is non-trivial to coordinate semantics modeling in SSL and redundancy removing in compression, so we design a novel information filtering (IF) module between them by co-optimization of instance distinguishment and entropy minimization to adaptively drop information that is weakly related to AI tasks (e.g., some texture redundancy). Different from previous task-specific solutions, Omni-ICM could directly support AI tasks analysis based on the learned omnipotent features without joint training or extra transformation. Albeit simple and intuitive, Omni-ICM significantly outperforms existing traditional and learning-based codecs on multiple fundamental vision tasks.