ICARUS: A Specialized Architecture for Neural Radiance Fields Rendering

TL;DR

ICARUS is a specialized hardware architecture designed to accelerate Neural Radiance Fields rendering, significantly improving speed and efficiency by executing the entire pipeline on dedicated cores without off-chip data exchanges.

Contribution

This paper introduces ICARUS, a novel FPGA-based accelerator architecture tailored specifically for NeRF rendering, replacing general-purpose GPU architectures with dedicated plenoptic cores.

Findings

ICARUS achieves 0.105 uJ/sample energy efficiency at 400 MHz.

The architecture occupies 16.5 mm^2 on 40nm CMOS technology.

Compared to GPU and TPU, ICARUS offers improved power-performance-area metrics.

Abstract

The practical deployment of Neural Radiance Fields (NeRF) in rendering applications faces several challenges, with the most critical one being low rendering speed on even high-end graphic processing units (GPUs). In this paper, we present ICARUS, a specialized accelerator architecture tailored for NeRF rendering. Unlike GPUs using general purpose computing and memory architectures for NeRF, ICARUS executes the complete NeRF pipeline using dedicated plenoptic cores (PLCore) consisting of a positional encoding unit (PEU), a multi-layer perceptron (MLP) engine, and a volume rendering unit (VRU). A PLCore takes in positions \& directions and renders the corresponding pixel colors without any intermediate data going off-chip for temporary storage and exchange, which can be time and power consuming. To implement the most expensive component of NeRF, i.e., the MLP, we transform the fully connected operations to approximated reconfigurable multiple constant multiplications (MCMs), where common subexpressions are shared across different multiplications to improve the computation efficiency. We build a prototype ICARUS using Synopsys HAPS-80 S104, a field programmable gate array (FPGA)-based prototyping system for large-scale integrated circuits and systems design. We evaluate the power-performance-area (PPA) of a PLCore using 40nm LP CMOS technology. Working at 400 MHz, a single PLCore occupies 16.5 $mm^2$ and consumes 282.8 mW, translating to 0.105 uJ/sample. The results are compared with those of GPU and tensor processing unit (TPU) implementations.