Compact Hadamard Latent Codes for Efficient Spectral Rendering

TL;DR

This paper introduces Hadamard spectral codes, a compact latent representation that enables efficient spectral rendering with standard RGB operations, reducing computational costs while maintaining high color accuracy.

Contribution

The authors propose a learned non-negative linear encoder-decoder architecture for spectral codes that preserve scaling and addition exactly, enabling spectral rendering with fewer RGB passes.

Findings

Using k=6 spectral codes reduces color error significantly.

Spectral rendering with these codes is faster than naive spectral methods.

A neural upsampling network allows legacy RGB assets to be integrated into the spectral pipeline.

Abstract

Spectral rendering accurately reproduces wavelength-dependent appearance but is computationally expensive, as shading must be evaluated at many wavelength samples and scales roughly linearly with the number of samples. It also requires spectral textures and lights throughout the rendering pipeline. We propose Hadamard spectral codes, a compact latent representation that enables spectral rendering using standard RGB rendering operations. Spectral images are approximated with a small number of RGB rendering passes, followed by a decoding step. Our key requirement is latent linearity: scaling and addition in spectral space correspond to scaling and addition of codes, and the element-wise product of spectra (for example reflectance times illumination) is approximated by the element-wise product of their latent codes. We show that an exact low-dimensional algebra-preserving representation cannot exist for arbitrary spectra when the latent dimension k is smaller than the number of spectral samples n. We therefore introduce a learned non-negative linear encoder and decoder architecture that preserves scaling and addition exactly while encouraging approximate multiplicativity under the Hadamard product. With k = 6, we render k/3 = 2 RGB images per frame using an unmodified RGB renderer, reconstruct the latent image, and decode to high-resolution spectra or XYZ or RGB. Experiments on 3D scenes demonstrate that k = 6 significantly reduces color error compared to RGB baselines while being substantially faster than naive n-sample spectral rendering. Using k = 9 provides higher-quality reference results. We further introduce a lightweight neural upsampling network that maps RGB assets directly to latent codes, enabling integration of legacy RGB content into the spectral pipeline while maintaining perceptually accurate colors in rendered images.