Pay Attention to Where You Looked

TL;DR

This paper introduces a camera-weighting mechanism for few-shot novel view synthesis that dynamically adjusts the importance of input views, significantly improving synthesis quality and realism.

Contribution

We propose a novel camera-weighting approach using geometric and attention-based schemes to enhance few-shot NVS performance.

Findings

Adaptive view weighting improves synthesis accuracy.

The method enhances realism in generated views.

Models trained with weighting scheme outperform baselines.

Abstract

Novel view synthesis (NVS) has advanced with generative modeling, enabling photorealistic image generation. In few-shot NVS, where only a few input views are available, existing methods often assume equal importance for all input views relative to the target, leading to suboptimal results. We address this limitation by introducing a camera-weighting mechanism that adjusts the importance of source views based on their relevance to the target. We propose two approaches: a deterministic weighting scheme leveraging geometric properties like Euclidean distance and angular differences, and a cross-attention-based learning scheme that optimizes view weighting. Additionally, models can be further trained with our camera-weighting scheme to refine their understanding of view relevance and enhance synthesis quality. This mechanism is adaptable and can be integrated into various NVS algorithms, improving their ability to synthesize high-quality novel views. Our results demonstrate that adaptive view weighting enhances accuracy and realism, offering a promising direction for improving NVS.