Soft Anisotropic Diagrams for Differentiable Image Representation

TL;DR

We propose Soft Anisotropic Diagrams (SAD), a differentiable image representation that enables efficient rendering, content-aligned boundaries, and superior performance on benchmarks, with significant speedups and compact storage.

Contribution

SAD introduces a novel soft anisotropic Voronoi-based image representation with GPU-friendly computation and adaptive training, outperforming existing methods in quality and speed.

Findings

SAD achieves 46.0 dB PSNR on Kodak at 2.2s encoding time.

Outperforms Image-GS and Instant-NGP at matched bitrate.

Provides 4-19x faster training speeds over state-of-the-art baselines.

Abstract

We introduce Soft Anisotropic Diagrams (SAD), an explicit and differentiable image representation parameterized by a set of adaptive sites in the image plane. In SAD, each site specifies an anisotropic metric and an additively weighted distance score, and we compute pixel colors as a softmax blend over a small per-pixel top-K subset of sites. We induce a soft anisotropic additively weighted Voronoi partition (i.e., an Apollonius diagram) with learnable per-site temperatures, preserving informative gradients while allowing clear, content-aligned boundaries and explicit ownership. Such a formulation enables efficient rendering by maintaining a per-query top-K map that approximates nearest neighbors under the same shading score, allowing GPU-friendly, fixed-size local computation. We update this list using our top-K propagation scheme inspired by jump flooding, augmented with stochastic injection to provide probabilistic global coverage. Training follows a GPU-first pipeline with gradient-weighted initialization, Adam optimization, and adaptive budget control through densification and pruning. Across standard benchmarks, SAD consistently outperforms Image-GS and Instant-NGP at matched bitrate. On Kodak, SAD reaches 46.0 dB PSNR with 2.2 s encoding time (vs. 28 s for Image-GS), and delivers 4-19 times end-to-end training speedups over state-of-the-art baselines. We demonstrate the effectiveness of SAD by showcasing the seamless integration with differentiable pipelines for forward and inverse problems, efficiency of fast random access, and compact storage.