Multiview Regenerative Morphing with Dual Flows

TL;DR

This paper introduces a novel multiview image morphing method that generates smooth, view-consistent transitions between scenes by learning volumetric representations and applying optimal transport, without requiring user correspondences or predefined categories.

Contribution

It proposes a new multiview regenerative morphing approach using volumetric modeling and optimal transport, enabling view-consistent scene transitions without prior correspondences.

Findings

Produces smooth, view-consistent scene transitions

Does not require user-specified correspondences

Works across arbitrary scene categories

Abstract

This paper aims to address a new task of image morphing under a multiview setting, which takes two sets of multiview images as the input and generates intermediate renderings that not only exhibit smooth transitions between the two input sets but also ensure visual consistency across different views at any transition state. To achieve this goal, we propose a novel approach called Multiview Regenerative Morphing that formulates the morphing process as an optimization to solve for rigid transformation and optimal-transport interpolation. Given the multiview input images of the source and target scenes, we first learn a volumetric representation that models the geometry and appearance for each scene to enable the rendering of novel views. Then, the morphing between the two scenes is obtained by solving optimal transport between the two volumetric representations in Wasserstein metrics. Our approach does not rely on user-specified correspondences or 2D/3D input meshes, and we do not assume any predefined categories of the source and target scenes. The proposed view-consistent interpolation scheme directly works on multiview images to yield a novel and visually plausible effect of multiview free-form morphing.