Intrinsic PAPR for Point-level 3D Scene Albedo and Shading Editing

TL;DR

This paper introduces Intrinsic PAPR, a novel point-based neural rendering method that enables detailed, 3D consistent editing of scene albedo and shading from multi-view RGB images, outperforming prior volumetric approaches.

Contribution

It proposes a new intrinsic decomposition approach within point-based neural rendering that avoids complex shading models, improving accuracy and editing capabilities.

Findings

Achieves higher-quality novel view rendering.

Enables superior point-level albedo and shading editing.

Outperforms recent inverse rendering methods.

Abstract

Recent advancements in neural rendering have excelled at novel view synthesis from multi-view RGB images. However, they often lack the capability to edit the shading or colour of the scene at a detailed point-level, while ensuring consistency across different viewpoints. In this work, we address the challenge of point-level 3D scene albedo and shading editing from multi-view RGB images, focusing on detailed editing at the point-level rather than at a part or global level. While prior works based on volumetric representation such as NeRF struggle with achieving 3D consistent editing at the point level, recent advancements in point-based neural rendering show promise in overcoming this challenge. We introduce ``Intrinsic PAPR'', a novel method based on the recent point-based neural rendering technique Proximity Attention Point Rendering (PAPR). Unlike other point-based methods that model the intrinsic decomposition of the scene, our approach does not rely on complicated shading models or simplistic priors that may not universally apply. Instead, we directly model scene decomposition into albedo and shading components, leading to better estimation accuracy. Comparative evaluations against the latest point-based inverse rendering methods demonstrate that Intrinsic PAPR achieves higher-quality novel view rendering and superior point-level albedo and shading editing.