Your Demands Deserve More Bits: Referring Semantic Image Compression at Ultra-low Bitrate
Chenhao Wu, Qingbo Wu, Haoran Wei, Shuai Chen, Mingzhou He, King Ngi Ngan, Fanman Meng, Hongliang Li

TL;DR
This paper introduces RSIC, a novel image compression framework that uses semantic guidance to improve fidelity of user-specified content at ultra-low bitrates, balancing realism and compression efficiency.
Contribution
The paper proposes a new RSIC framework with modules for semantic encoding and guided generative decoding, enhancing fidelity and flexibility in ultra-low bitrate image compression.
Findings
RSIC achieves superior compression efficiency on three datasets.
RSIC provides flexible, user-demanded content reconstruction.
RSIC maintains high semantic alignment at ultra-low bitrates.
Abstract
With the help of powerful generative models, Semantic Image Compression (SIC) has achieved impressive performance at ultra-low bitrate. However, due to coarse-grained visual-semantic alignment and inherent randomness, the reliability of SIC is seriously concerned for reconstructing completely different object instances, even they are semantically consistent with original images. To tackle this issue, we propose a novel Referring Semantic Image Compression (RSIC) framework to improve the fidelity of user-specified content while retaining extreme compression ratios. Specifically, RSIC consists of three modules: Global Description Encoding (GDE), Referring Guidance Encoding (RGE), and Guided Generative Decoding (GGD). GDE and RGE encode global semantic information and local features, respectively, while GGD handles the non-uniformly guided generative process based on the encoded…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Data Compression Techniques · Image Retrieval and Classification Techniques · Video Analysis and Summarization
