Evolutive Rendering Models

TL;DR

This paper introduces evolutive rendering models that adapt dynamically during rendering, optimizing key elements through differentiable frameworks to improve performance in static, dynamic, and generative scene rendering.

Contribution

It pioneers a comprehensive learning framework for evolving rendering models by differentiably optimizing gauge transformations, sampling mechanisms, and primitive organization.

Findings

Enhanced rendering performance across multiple domains

Effective gradient-based optimization of rendering elements

Potential for improved static and dynamic scene rendering

Abstract

The landscape of computer graphics has undergone significant transformations with the recent advances of differentiable rendering models. These rendering models often rely on heuristic designs that may not fully align with the final rendering objectives. We address this gap by pioneering \textit{evolutive rendering models}, a methodology where rendering models possess the ability to evolve and adapt dynamically throughout the rendering process. In particular, we present a comprehensive learning framework that enables the optimization of three principal rendering elements, including the gauge transformations, the ray sampling mechanisms, and the primitive organization. Central to this framework is the development of differentiable versions of these rendering elements, allowing for effective gradient backpropagation from the final rendering objectives. A detailed analysis of gradient characteristics is performed to facilitate a stable and goal-oriented elements evolution. Our extensive experiments demonstrate the large potential of evolutive rendering models for enhancing the rendering performance across various domains, including static and dynamic scene representations, generative modeling, and texture mapping.