# NRMVS: Non-Rigid Multi-View Stereo

## TL;DR

This paper introduces NRMVS, a novel method for dense 3D reconstruction of non-rigid scenes from sparse, wide-baseline RGB images, addressing a key limitation of traditional multi-view stereo techniques.

## Contribution

It formulates non-rigid scene reconstruction as joint deformation and depth estimation using deformation graphs, with new matching and evaluation schemes.

## Key findings

- Successful dense 4D reconstruction from sparse images
- Effective interpolation of novel deformed scenes
- Demonstrated applicability to real-world non-rigid scenes

## Abstract

Scene reconstruction from unorganized RGB images is an important task in many computer vision applications. Multi-view Stereo (MVS) is a common solution in photogrammetry applications for the dense reconstruction of a static scene. The static scene assumption, however, limits the general applicability of MVS algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes, faces, or human bodies. In this paper, we open up a new challenging direction: dense 3D reconstruction of scenes with non-rigid changes observed from arbitrary, sparse, and wide-baseline views. We formulate the problem as a joint optimization of deformation and depth estimation, using deformation graphs as the underlying representation. We propose a new sparse 3D to 2D matching technique, together with a dense patch-match evaluation scheme to estimate deformation and depth with photometric consistency. We show that creating a dense 4D structure from a few RGB images with non-rigid changes is possible, and demonstrate that our method can be used to interpolate novel deformed scenes from various combinations of these deformation estimates derived from the sparse views.