Activation Map-based Vector Quantization for 360-degree Image Semantic Communication

TL;DR

This paper introduces AM-VQ, a novel neural network-based vector quantization method that adaptively compresses 360-degree images for VR, reducing data size while maintaining high quality through adversarial training.

Contribution

The paper proposes a new activation map-based vector quantization framework that improves semantic feature compression for 360-degree images in VR applications.

Findings

Outperforms existing DL-based coding schemes

Reduces communication overhead in wireless transmission

Enhances image reconstruction quality with adversarial training

Abstract

In virtual reality (VR) applications, 360-degree images play a pivotal role in crafting immersive experiences and offering panoramic views, thus improving user Quality of Experience (QoE). However, the voluminous data generated by 360-degree images poses challenges in network storage and bandwidth. To address these challenges, we propose a novel Activation Map-based Vector Quantization (AM-VQ) framework, which is designed to reduce communication overhead for wireless transmission. The proposed AM-VQ scheme uses the Deep Neural Networks (DNNs) with vector quantization (VQ) to extract and compress semantic features. Particularly, the AM-VQ framework utilizes activation map to adaptively quantize semantic features, thus reducing data distortion caused by quantization operation. To further enhance the reconstruction quality of the 360-degree image, adversarial training with a Generative Adversarial Networks (GANs) discriminator is incorporated. Numerical results show that our proposed AM-VQ scheme achieves better performance than the existing Deep Learning (DL) based coding and the traditional coding schemes under the same transmission symbols.