GenDR: A Generalized Differentiable Renderer

TL;DR

This paper introduces a generalized family of differentiable renderers that unify existing methods and explore various smoothing distributions, providing insights into their performance across 3D reconstruction tasks.

Contribution

It formalizes the components of differentiable rendering, proposes a generalized renderer with multiple smoothing options, and evaluates their effectiveness on ShapeNet.

Findings

Uniform distribution performs best overall

Optimal distribution choice depends on specific task

Different distributions impact reconstruction quality

Abstract

In this work, we present and study a generalized family of differentiable renderers. We discuss from scratch which components are necessary for differentiable rendering and formalize the requirements for each component. We instantiate our general differentiable renderer, which generalizes existing differentiable renderers like SoftRas and DIB-R, with an array of different smoothing distributions to cover a large spectrum of reasonable settings. We evaluate an array of differentiable renderer instantiations on the popular ShapeNet 3D reconstruction benchmark and analyze the implications of our results. Surprisingly, the simple uniform distribution yields the best overall results when averaged over 13 classes; in general, however, the optimal choice of distribution heavily depends on the task.