Sparse Point Clouds Assisted Learned Image Compression

TL;DR

This paper introduces a novel method that leverages sparse point clouds from autonomous driving sensors to improve learned image compression by predicting images and extracting structural features, resulting in enhanced compression performance.

Contribution

It presents a new framework that integrates sparse point cloud data into learned image compression, demonstrating consistent performance improvements across various models.

Findings

Enhanced compression performance with point cloud assistance

Framework compatible with multiple learned image compression models

Validated improvements through extensive experiments

Abstract

In the field of autonomous driving, a variety of sensor data types exist, each representing different modalities of the same scene. Therefore, it is feasible to utilize data from other sensors to facilitate image compression. However, few techniques have explored the potential benefits of utilizing inter-modality correlations to enhance the image compression performance. In this paper, motivated by the recent success of learned image compression, we propose a new framework that uses sparse point clouds to assist in learned image compression in the autonomous driving scenario. We first project the 3D sparse point cloud onto a 2D plane, resulting in a sparse depth map. Utilizing this depth map, we proceed to predict camera images. Subsequently, we use these predicted images to extract multi-scale structural features. These features are then incorporated into learned image compression pipeline as additional information to improve the compression performance. Our proposed framework is compatible with various mainstream learned image compression models, and we validate our approach using different existing image compression methods. The experimental results show that incorporating point cloud assistance into the compression pipeline consistently enhances the performance.