UGAE: Unified Geometry and Attribute Enhancement for G-PCC Compressed Point Clouds

TL;DR

The paper introduces UGAE, a comprehensive framework that enhances lossy compressed point clouds by improving geometry and attribute quality through innovative deep learning strategies, significantly outperforming existing methods.

Contribution

UGAE is the first unified framework combining geometry and attribute enhancement for G-PCC compressed point clouds, utilizing Transformer-based reconstruction and detail-aware recoloring.

Findings

Achieved an average BD-PSNR gain of 9.98 dB for geometry.

Reduced BD-bitrate by 90.98% for geometry compression.

Improved attribute quality with a 3.67 dB BD-PSNR increase.

Abstract

Lossy compression of point clouds reduces storage and transmission costs; however, it inevitably leads to irreversible distortion in geometry structure and attribute information. To address these issues, we propose a unified geometry and attribute enhancement (UGAE) framework, which consists of three core components: post-geometry enhancement (PoGE), pre-attribute enhancement (PAE), and post-attribute enhancement (PoAE). In PoGE, a Transformer-based sparse convolutional U-Net is used to reconstruct the geometry structure with high precision by predicting voxel occupancy probabilities. Building on the refined geometry structure, PAE introduces an innovative enhanced geometry-guided recoloring strategy, which uses a detail-aware K-Nearest Neighbors (DA-KNN) method to achieve accurate recoloring and effectively preserve high-frequency details before attribute compression. Finally, at the decoder side, PoAE uses an attribute residual prediction network with a weighted mean squared error (W-MSE) loss to enhance the quality of high-frequency regions while maintaining the fidelity of low-frequency regions. UGAE significantly outperformed existing methods on three benchmark datasets: 8iVFB, Owlii, and MVUB. Compared to the latest G-PCC test model (TMC13v29), UGAE achieved an average BD-PSNR gain of 9.98 dB and 90.98% BD-bitrate savings for geometry under the D1 metric, as well as a 3.67 dB BD-PSNR improvement with 56.88% BD-bitrate savings for attributes on the Y component. Additionally, it improved perceptual quality significantly.