Potamoi: Accelerating Neural Rendering via a Unified Streaming Architecture

TL;DR

Potamoi is a unified system that accelerates neural rendering by optimizing algorithm and hardware design, achieving real-time performance with significant speed and energy efficiency improvements while maintaining high visual quality.

Contribution

The paper introduces Potamoi, a novel unified streaming architecture with a plug-and-play algorithm and hardware support, addressing bottlenecks in NeRF rendering for resource-constrained devices.

Findings

53.1× speed-up over baseline

67.7× energy reduction

Less than 1.0 dB PSNR loss

Abstract

Neural Radiance Field (NeRF) has emerged as a promising alternative for photorealistic rendering. Despite recent algorithmic advancements, achieving real-time performance on today's resource-constrained devices remains challenging. In this paper, we identify the primary bottlenecks in current NeRF algorithms and introduce a unified algorithm-architecture co-design, Potamoi, designed to accommodate various NeRF algorithms. Specifically, we introduce a runtime system featuring a plug-and-play algorithm, SpaRW, which significantly reduces the per-frame computational workload and alleviates compute inefficiencies. Furthermore, our unified streaming pipeline coupled with customized hardware support effectively tames both SRAM and DRAM inefficiencies by minimizing repetitive DRAM access and completely eliminating SRAM bank conflicts. When evaluated against a baseline utilizing a dedicated DNN accelerator, our framework demonstrates a speed-up and energy reduction of 53.1$\times$ and 67.7$\times$, respectively, all while maintaining high visual quality with less than a 1.0 dB reduction in peak signal-to-noise ratio.