NeRF View Synthesis: Subjective Quality Assessment and Objective Metrics Evaluation

TL;DR

This paper thoroughly evaluates the quality of NeRF view synthesis through subjective assessments and compares various objective metrics, highlighting the importance of pose correction for accurate quality measurement.

Contribution

It provides the first comprehensive subjective quality assessment for NeRF view synthesis and evaluates the effectiveness of existing objective metrics in this context.

Findings

Errors in camera pose estimation affect quality assessment accuracy.

Objective metrics vary in correlation with subjective scores.

Scene type influences the performance of quality assessment methods.

Abstract

Neural radiance fields (NeRF) are a groundbreaking computer vision technology that enables the generation of high-quality, immersive visual content from multiple viewpoints. This capability has significant advantages for applications such as virtual/augmented reality, 3D modelling, and content creation for the film and entertainment industry. However, the evaluation of NeRF methods poses several challenges, including a lack of comprehensive datasets, reliable assessment methodologies, and objective quality metrics. This paper addresses the problem of NeRF view synthesis (NVS) quality assessment thoroughly, by conducting a rigorous subjective quality assessment test that considers several scene classes and recently proposed NVS methods. Additionally, the performance of a wide range of state-of-the-art conventional and learning-based full-reference 2D image and video quality assessment metrics is evaluated against the subjective scores of the subjective study. This study found that errors in camera pose estimation can result in spatial misalignments between synthesized and reference images, which need to be corrected before applying an objective quality metric. The experimental results are analyzed in depth, providing a comparative evaluation of several NVS methods and objective quality metrics, across different classes of visual scenes, including real and synthetic content for front-face and 360-degree camera trajectories.