TL;DR
Stable View Synthesis (SVS) is a novel end-to-end method that synthesizes realistic new views of a scene from multiple source images by leveraging a geometric scaffold and view-dependent feature aggregation, outperforming existing techniques.
Contribution
SVS introduces a differentiable, end-to-end approach that combines structure-from-motion with view-dependent feature processing for high-quality view synthesis.
Findings
Outperforms state-of-the-art methods quantitatively.
Achieves high realism in free-viewpoint videos.
Supports synthesis of view-dependent effects like specular reflection.
Abstract
We present Stable View Synthesis (SVS). Given a set of source images depicting a scene from freely distributed viewpoints, SVS synthesizes new views of the scene. The method operates on a geometric scaffold computed via structure-from-motion and multi-view stereo. Each point on this 3D scaffold is associated with view rays and corresponding feature vectors that encode the appearance of this point in the input images. The core of SVS is view-dependent on-surface feature aggregation, in which directional feature vectors at each 3D point are processed to produce a new feature vector for a ray that maps this point into the new target view. The target view is then rendered by a convolutional network from a tensor of features synthesized in this way for all pixels. The method is composed of differentiable modules and is trained end-to-end. It supports spatially-varying view-dependent…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
