Conceptual Compression via Deep Structure and Texture Synthesis

TL;DR

This paper introduces a novel conceptual compression method that encodes images into structure and texture components, enabling high-quality reconstruction, flexible manipulation, and supporting various vision tasks with lower bitrates.

Contribution

It proposes a dual-layered model for image compression using structure and texture representations, combined with a hierarchical fusion GAN for synthesis, improving quality and versatility.

Findings

Lower bitrates achieved compared to traditional methods

Higher reconstruction quality with realistic visuals

Enhanced versatility for visual analysis and manipulation

Abstract

Existing compression methods typically focus on the removal of signal-level redundancies, while the potential and versatility of decomposing visual data into compact conceptual components still lack further study. To this end, we propose a novel conceptual compression framework that encodes visual data into compact structure and texture representations, then decodes in a deep synthesis fashion, aiming to achieve better visual reconstruction quality, flexible content manipulation, and potential support for various vision tasks. In particular, we propose to compress images by a dual-layered model consisting of two complementary visual features: 1) structure layer represented by structural maps and 2) texture layer characterized by low-dimensional deep representations. At the encoder side, the structural maps and texture representations are individually extracted and compressed, generating the compact, interpretable, inter-operable bitstreams. During the decoding stage, a hierarchical fusion GAN (HF-GAN) is proposed to learn the synthesis paradigm where the textures are rendered into the decoded structural maps, leading to high-quality reconstruction with remarkable visual realism. Extensive experiments on diverse images have demonstrated the superiority of our framework with lower bitrates, higher reconstruction quality, and increased versatility towards visual analysis and content manipulation tasks.