Pay Attention to Where You Looked
Alex Berian, JhihYang Wu, Daniel Brignac, Natnael Daba, Abhijit Mahalanobis

TL;DR
This paper introduces a camera-weighting mechanism for few-shot novel view synthesis that dynamically adjusts the importance of input views, significantly improving synthesis quality and realism.
Contribution
We propose a novel camera-weighting approach using geometric and attention-based schemes to enhance few-shot NVS performance.
Findings
Adaptive view weighting improves synthesis accuracy.
The method enhances realism in generated views.
Models trained with weighting scheme outperform baselines.
Abstract
Novel view synthesis (NVS) has advanced with generative modeling, enabling photorealistic image generation. In few-shot NVS, where only a few input views are available, existing methods often assume equal importance for all input views relative to the target, leading to suboptimal results. We address this limitation by introducing a camera-weighting mechanism that adjusts the importance of source views based on their relevance to the target. We propose two approaches: a deterministic weighting scheme leveraging geometric properties like Euclidean distance and angular differences, and a cross-attention-based learning scheme that optimizes view weighting. Additionally, models can be further trained with our camera-weighting scheme to refine their understanding of view relevance and enhance synthesis quality. This mechanism is adaptable and can be integrated into various NVS algorithms,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Image Enhancement Techniques · Generative Adversarial Networks and Image Synthesis
