Active View Selector: Fast and Accurate Active View Selection with Cross Reference Image Quality Assessment

TL;DR

This paper introduces a fast, 2D image quality assessment-based method for active view selection in 3D reconstruction, outperforming existing 3D-aware methods in accuracy and speed by leveraging cross-reference IQA models.

Contribution

It reframes active view selection as a 2D IQA problem using cross-reference models, enabling faster and more versatile view selection across different 3D representations.

Findings

Achieves significant quantitative improvements on benchmarks.

Runs 14-33 times faster than previous methods.

Is agnostic to 3D representation types.

Abstract

We tackle active view selection in novel view synthesis and 3D reconstruction. Existing methods like FisheRF and ActiveNeRF select the next best view by minimizing uncertainty or maximizing information gain in 3D, but they require specialized designs for different 3D representations and involve complex modelling in 3D space. Instead, we reframe this as a 2D image quality assessment (IQA) task, selecting views where current renderings have the lowest quality. Since ground-truth images for candidate views are unavailable, full-reference metrics like PSNR and SSIM are inapplicable, while no-reference metrics, such as MUSIQ and MANIQA, lack the essential multi-view context. Inspired by a recent cross-referencing quality framework CrossScore, we train a model to predict SSIM within a multi-view setup and use it to guide view selection. Our cross-reference IQA framework achieves substantial quantitative and qualitative improvements across standard benchmarks, while being agnostic to 3D representations, and runs 14-33 times faster than previous methods.