Cross-architecture universal feature coding via distribution alignment

TL;DR

This paper introduces a novel approach for cross-architecture universal feature coding that unifies CNN and Transformer features through distribution alignment, enabling effective compression across different model architectures.

Contribution

The paper proposes the first method for universal feature coding across CNNs and Transformers using a two-step distribution alignment process.

Findings

Achieves better rate-accuracy trade-offs than architecture-specific methods.

Unifies feature formats from CNNs and Transformers successfully.

Demonstrates potential for universal feature compression in image classification.

Abstract

Feature coding has become increasingly important in scenarios where semantic representations rather than raw pixels are transmitted and stored. However, most existing methods are architecture-specific, targeting either CNNs or Transformers. This design limits their applicability in real-world scenarios where features from both architectures coexist. To address this gap, we introduce a new research problem: cross-architecture universal feature coding (CAUFC), which seeks to build a unified codec that can effectively compress features from heterogeneous architectures. To tackle this challenge, we propose a two-step distribution alignment method. First, we design the format alignment method that unifies CNN and Transformer features into a consistent 2D token format. Second, we propose the feature value alignment method that harmonizes statistical distributions via truncation and normalization. As a first attempt to study CAUFC, we evaluate our method on the image classification task. Experimental results demonstrate that our method achieves superior rate-accuracy trade-offs compared to the architecture-specific baseline. This work marks an initial step toward universal feature compression across heterogeneous model architectures.