FlexNeRFer: A Multi-Dataflow, Adaptive Sparsity-Aware Accelerator for On-Device NeRF Rendering

TL;DR

FlexNeRFer is a versatile, energy-efficient hardware accelerator designed for on-device NeRF rendering, supporting multiple models and dataflows, significantly outperforming GPUs and existing accelerators in speed and energy efficiency.

Contribution

The paper introduces FlexNeRFer, a novel adaptive NeRF accelerator with a flexible NoC and optimized sparsity data storage, enhancing efficiency across diverse NeRF models.

Findings

Achieves up to 243.3x speedup over GPU

Provides 520.3x energy efficiency improvement over GPU

Outperforms state-of-the-art NeRF accelerator NeuRex by up to 86.9x speedup

Abstract

Neural Radiance Fields (NeRF), an AI-driven approach for 3D view reconstruction, has demonstrated impressive performance, sparking active research across fields. As a result, a range of advanced NeRF models has emerged, leading on-device applications to increasingly adopt NeRF for highly realistic scene reconstructions. With the advent of diverse NeRF models, NeRF-based applications leverage a variety of NeRF frameworks, creating the need for hardware capable of efficiently supporting these models. However, GPUs fail to meet the performance, power, and area (PPA) cost demanded by these on-device applications, or are specialized for specific NeRF algorithms, resulting in lower efficiency when applied to other NeRF models. To address this limitation, in this work, we introduce FlexNeRFer, an energy-efficient versatile NeRF accelerator. The key components enabling the enhancement of FlexNeRFer include: i) a flexible network-on-chip (NoC) supporting multi-dataflow and sparsity on precision-scalable MAC array, and ii) efficient data storage using an optimal sparsity format based on the sparsity ratio and precision modes. To evaluate the effectiveness of FlexNeRFer, we performed a layout implementation using 28nm CMOS technology. Our evaluation shows that FlexNeRFer achieves 8.2~243.3x speedup and 24.1~520.3x improvement in energy efficiency over a GPU (i.e., NVIDIA RTX 2080 Ti), while demonstrating 4.2~86.9x speedup and 2.3~47.5x improvement in energy efficiency compared to a state-of-the-art NeRF accelerator (i.e., NeuRex).