End-to-End RGB-IR Joint Image Compression With Channel-wise Cross-modality Entropy Model

TL;DR

This paper introduces a novel joint compression framework for RGB-IR image pairs that leverages cross-modality information to improve compression efficiency, significantly reducing bit rates while maintaining image quality.

Contribution

It proposes a Channel-wise Cross-modality Entropy Model (CCEM) with specialized blocks for low-frequency information extraction and fusion, enhancing entropy modeling across modalities.

Findings

Achieves 23.1% bit rate savings on LLVIP dataset.

Outperforms existing RGB-IR and single-modality compression methods.

Demonstrates improved compression efficiency with cross-modality modeling.

Abstract

RGB-IR(RGB-Infrared) image pairs are frequently applied simultaneously in various applications like intelligent surveillance. However, as the number of modalities increases, the required data storage and transmission costs also double. Therefore, efficient RGB-IR data compression is essential. This work proposes a joint compression framework for RGB-IR image pair. Specifically, to fully utilize cross-modality prior information for accurate context probability modeling within and between modalities, we propose a Channel-wise Cross-modality Entropy Model (CCEM). Among CCEM, a Low-frequency Context Extraction Block (LCEB) and a Low-frequency Context Fusion Block (LCFB) are designed for extracting and aggregating the global low-frequency information from both modalities, which assist the model in predicting entropy parameters more accurately. Experimental results demonstrate that our approach outperforms existing RGB-IR image pair and single-modality compression methods on LLVIP and KAIST datasets. For instance, the proposed framework achieves a 23.1% bit rate saving on LLVIP dataset compared to the state-of-the-art RGB-IR image codec presented at CVPR 2022.