DiffBMP: Differentiable Rendering with Bitmap Primitives

TL;DR

DiffBMP is a scalable, efficient differentiable rendering engine for bitmap images that enables fast optimization of bitmap primitives using GPU acceleration, facilitating creative workflows and image editing tasks.

Contribution

The paper introduces DiffBMP, a novel GPU-accelerated differentiable renderer for bitmap images, expanding differentiable rendering capabilities beyond vector graphics.

Findings

Optimizes thousands of bitmap primitives in under 1 minute.

Supports exporting layered compositions for creative workflows.

Provides a publicly accessible Python package for easy integration.

Abstract

We introduce DiffBMP, a scalable and efficient differentiable rendering engine for a collection of bitmap images. Our work addresses a limitation that traditional differentiable renderers are constrained to vector graphics, given that most images in the world are bitmaps. Our core contribution is a highly parallelized rendering pipeline, featuring a custom CUDA implementation for calculating gradients. This system can, for example, optimize the position, rotation, scale, color, and opacity of thousands of bitmap primitives all in under 1 min using a consumer GPU. We employ and validate several techniques to facilitate the optimization: soft rasterization via Gaussian blur, structure-aware initialization, noisy canvas, and specialized losses/heuristics for videos or spatially constrained images. We demonstrate DiffBMP is not just an isolated tool, but a practical one designed to integrate into creative workflows. It supports exporting compositions to a native, layered file format, and the entire framework is publicly accessible via an easy-to-hack Python package.