From Images to Point Clouds: An Efficient Solution for Cross-media Blind Quality Assessment without Annotated Training

TL;DR

This paper introduces DWIT-PCQA, a novel method that predicts point cloud quality without annotations by transferring image-based perceptual knowledge through a distortion-aware domain adaptation framework.

Contribution

It proposes a distortion-guided biased feature alignment and quality-aware feature disentanglement to improve cross-media quality assessment without annotated training data.

Findings

Achieves reliable point cloud quality prediction without point cloud annotations.

Outperforms existing blind PCQA methods in experimental evaluations.

Effectively leverages image priors for point cloud quality assessment.

Abstract

We present a novel quality assessment method which can predict the perceptual quality of point clouds from new scenes without available annotations by leveraging the rich prior knowledge in images, called the Distribution-Weighted Image-Transferred Point Cloud Quality Assessment (DWIT-PCQA). Recognizing the human visual system (HVS) as the decision-maker in quality assessment regardless of media types, we can emulate the evaluation criteria for human perception via neural networks and further transfer the capability of quality prediction from images to point clouds by leveraging the prior knowledge in the images. Specifically, domain adaptation (DA) can be leveraged to bridge the images and point clouds by aligning feature distributions of the two media in the same feature space. However, the different manifestations of distortions in images and point clouds make feature alignment a difficult task. To reduce the alignment difficulty and consider the different distortion distribution during alignment, we have derived formulas to decompose the optimization objective of the conventional DA into two suboptimization functions with distortion as a transition. Specifically, through network implementation, we propose the distortion-guided biased feature alignment which integrates existing/estimated distortion distribution into the adversarial DA framework, emphasizing common distortion patterns during feature alignment. Besides, we propose the quality-aware feature disentanglement to mitigate the destruction of the mapping from features to quality during alignment with biased distortions. Experimental results demonstrate that our proposed method exhibits reliable performance compared to general blind PCQA methods without needing point cloud annotations.