Neural Mesh-Based Graphics

TL;DR

This paper introduces a mesh-based neural rendering approach that enhances data efficiency and view synthesis speed, outperforming existing methods trained on multiple scenes by training on a single scene.

Contribution

It proposes a novel view-dependent mesh rasterization and scene split technique, achieving superior results with less training data and computational resources.

Findings

Outperforms NPBG trained on multiple scenes using only a single scene.

Achieves competitive results with state-of-the-art methods trained on full datasets.

Improves data efficiency and rendering speed in neural scene synthesis.

Abstract

We revisit NPBG, the popular approach to novel view synthesis that introduced the ubiquitous point feature neural rendering paradigm. We are interested in particular in data-efficient learning with fast view synthesis. We achieve this through a view-dependent mesh-based denser point descriptor rasterization, in addition to a foreground/background scene rendering split, and an improved loss. By training solely on a single scene, we outperform NPBG, which has been trained on ScanNet and then scene finetuned. We also perform competitively with respect to the state-of-the-art method SVS, which has been trained on the full dataset (DTU and Tanks and Temples) and then scene finetuned, in spite of their deeper neural renderer.