Point Cloud-Assisted Neural Image Compression

TL;DR

This paper introduces PCA-NIC, a novel neural image codec that leverages point cloud data to improve image compression efficiency and quality, especially in autonomous driving scenarios.

Contribution

It proposes a unified data representation, a multi-modal feature fusion transform module, and demonstrates state-of-the-art performance in point cloud-assisted image compression.

Findings

Achieves superior compression performance over existing methods.

Effectively preserves image texture and structure using point cloud data.

Introduces a novel multi-modal feature fusion module for enhanced feature extraction.

Abstract

High-efficient image compression is a critical requirement. In several scenarios where multiple modalities of data are captured by different sensors, the auxiliary information from other modalities are not fully leveraged by existing image-only codecs, leading to suboptimal compression efficiency. In this paper, we increase image compression performance with the assistance of point cloud, which is widely adopted in the area of autonomous driving. We first unify the data representation for both modalities to facilitate data processing. Then, we propose the point cloud-assisted neural image codec (PCA-NIC) to enhance the preservation of image texture and structure by utilizing the high-dimensional point cloud information. We further introduce a multi-modal feature fusion transform module (MMFFT) to capture more representative image features, remove redundant information between channels and modalities that are not relevant to the image content. Our work is the first to improve image compression performance using point cloud and achieves state-of-the-art performance.