Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer

TL;DR

This paper introduces a novel deep learning framework that optimizes 3D point cloud attribute compression by directly considering the final rendered image quality, utilizing a sparse tensor transformer for improved feature analysis.

Contribution

The paper presents a rendering-oriented point cloud attribute compression method that integrates differentiable rendering with a sparse tensor transformer, advancing beyond traditional reconstruction-focused approaches.

Findings

Achieves state-of-the-art compression performance.

Effectively captures point cloud relationships with sparse tensor transformer.

Improves rendered image quality through end-to-end optimization.

Abstract

The evolution of 3D visualization techniques has fundamentally transformed how we interact with digital content. At the forefront of this change is point cloud technology, offering an immersive experience that surpasses traditional 2D representations. However, the massive data size of point clouds presents significant challenges in data compression. Current methods for lossy point cloud attribute compression (PCAC) generally focus on reconstructing the original point clouds with minimal error. However, for point cloud visualization scenarios, the reconstructed point clouds with distortion still need to undergo a complex rendering process, which affects the final user-perceived quality. In this paper, we propose an end-to-end deep learning framework that seamlessly integrates PCAC with differentiable rendering, denoted as rendering-oriented PCAC (RO-PCAC), directly targeting the quality of rendered multiview images for viewing. In a differentiable manner, the impact of the rendering process on the reconstructed point clouds is taken into account. Moreover, we characterize point clouds as sparse tensors and propose a sparse tensor-based transformer, called SP-Trans. By aligning with the local density of the point cloud and utilizing an enhanced local attention mechanism, SP-Trans captures the intricate relationships within the point cloud, further improving feature analysis and synthesis within the framework. Extensive experiments demonstrate that the proposed RO-PCAC achieves state-of-the-art compression performance, compared to existing reconstruction-oriented methods, including traditional, learning-based, and hybrid methods.