Semantic-Aware Image Compressed Sensing

TL;DR

This paper introduces a semantic-aware image compressed sensing system that adaptively determines measurement matrices based on image content, improving sensing efficiency and reconstruction quality over traditional fixed-matrix methods.

Contribution

The paper presents a novel semantic-aware CS system with a policy network for adaptive measurement matrix selection and a specialized reconstruction network, trained end-to-end with a rate-distortion loss.

Findings

Outperforms traditional fixed-matrix CS systems in numerical tests.

Adaptive measurement matrix selection improves sensing efficiency.

Joint training with rate-distortion loss enhances reconstruction quality.

Abstract

Deep learning based image compressed sensing (CS) has achieved great success. However, existing CS systems mainly adopt a fixed measurement matrix to images, ignoring the fact the optimal measurement numbers and bases are different for different images. To further improve the sensing efficiency, we propose a novel semantic-aware image CS system. In our system, the encoder first uses a fixed number of base CS measurements to sense different images. According to the base CS results, the encoder then employs a policy network to analyze the semantic information in images and determines the measurement matrix for different image areas. At the decoder side, a semantic-aware initial reconstruction network is developed to deal with the changes of measurement matrices used at the encoder. A rate-distortion training loss is further introduced to dynamically adjust the average compression ratio for the semantic-aware CS system and the policy network is trained jointly with the encoder and the decoder in an en-to-end manner by using some proxy functions. Numerical results show that the proposed semantic-aware image CS system is superior to the traditional ones with fixed measurement matrices.