Hardware Accelerated Neural Block Texture Compression with Cooperative Vectors

TL;DR

This paper introduces a hardware-accelerated neural texture compression method that leverages existing block compression formats and hardware matrix engines to efficiently render high-quality 4K texture sets with minimal VRAM usage.

Contribution

It extends neural texture compression by integrating hardware acceleration and low dynamic range formats, enabling higher quality or compression ratios with real-time performance.

Findings

Achieves 4K texture rendering at 0.55ms on an Intel B580

Uses only 28MB VRAM for 9-channel textures at 1080p

Demonstrates improved performance with hardware matrix multiplication

Abstract

In this work, we present an extension to the neural texture compression method of Weinreich and colleagues [2024]. Like them, we leverage existing block compression methods which permit to use hardware texture filtering to store a neural representation of physically-based rendering (PBR) texture sets (including albedo, normal maps, roughness, etc.). However, we show that low dynamic range block compression formats still make the solution viable. Thanks to this, we show that we can achieve higher compression ratio or higher quality at fixed compression ratio. We improve performance at runtime using a tile based rendering architecture that leverage hardware matrix multiplication engine. Thanks to all this, we render 4k textures sets (9 channels per asset) with anisotropic filtering at 1080p using only 28MB of VRAM per texture set at 0.55ms on an Intel B580.