Non-Aligned Reference Image Quality Assessment for Novel View Synthesis

TL;DR

This paper introduces a novel Non-Aligned Reference IQA framework for evaluating the perceptual quality of NVS images without pixel-level alignment, leveraging synthetic distortions and contrastive learning to outperform existing methods.

Contribution

The work presents a new NAR-IQA model tailored for non-aligned references in NVS, utilizing synthetic data and contrastive learning to improve generalization and performance.

Findings

Outperforms state-of-the-art IQA methods on aligned and non-aligned references.

Achieves strong correlation with human subjective ratings.

Demonstrates robustness without overfitting to real NVS samples.

Abstract

Evaluating the perceptual quality of Novel View Synthesis (NVS) images remains a key challenge, particularly in the absence of pixel-aligned ground truth references. Full-Reference Image Quality Assessment (FR-IQA) methods fail under misalignment, while No-Reference (NR-IQA) methods struggle with generalization. In this work, we introduce a Non-Aligned Reference (NAR-IQA) framework tailored for NVS, where it is assumed that the reference view shares partial scene content but lacks pixel-level alignment. We constructed a large-scale image dataset containing synthetic distortions targeting Temporal Regions of Interest (TROI) to train our NAR-IQA model. Our model is built on a contrastive learning framework that incorporates LoRA-enhanced DINOv2 embeddings and is guided by supervision from existing IQA methods. We train exclusively on synthetically generated distortions, deliberately avoiding overfitting to specific real NVS samples and thereby enhancing the model's generalization capability. Our model outperforms state-of-the-art FR-IQA, NR-IQA, and NAR-IQA methods, achieving robust performance on both aligned and non-aligned references. We also conducted a novel user study to gather data on human preferences when viewing non-aligned references in NVS. We find strong correlation between our proposed quality prediction model and the collected subjective ratings. For dataset and code, please visit our project page: https://stootaghaj.github.io/nova-project/