ASDR: Exploiting Adaptive Sampling and Data Reuse for CIM-based Instant Neural Rendering

TL;DR

This paper introduces ASDR, a CIM-based accelerator that combines adaptive sampling and data reuse to significantly speed up neural rendering with minimal quality loss.

Contribution

It presents a novel co-designed algorithm-architecture approach that optimizes neural rendering for efficiency and speed using CIM technology.

Findings

Achieves up to 9.55x speedup over state-of-the-art NeRF accelerators.

Achieves up to 69.75x speedup over Xavier NX GPU.

Maintains only 0.1 PSNR loss in rendering quality.

Abstract

Neural Radiance Fields (NeRF) offer significant promise for generating photorealistic images and videos. However, existing mainstream neural rendering models often fall short in meeting the demands for immediacy and power efficiency in practical applications. Specifically, these models frequently exhibit irregular access patterns and substantial computational overhead, leading to undesirable inference latency and high power consumption. Computing-in-memory (CIM), an emerging computational paradigm, has the potential to address these access bottlenecks and reduce the power consumption associated with model execution. To bridge the gap between model performance and real-world scene requirements, we propose an algorithm-architecture co-design approach, abbreviated as ASDR, a CIM-based accelerator supporting efficient neural rendering. At the algorithmic level, we propose two rendering optimization schemes: (1) Dynamic sampling by online sensing of the rendering difficulty of different pixels, thus reducing access memory and computational overhead. (2) Reducing MLP overhead by decoupling and approximating the volume rendering of color and density. At the architecture level, we design an efficient ReRAM-based CIM architecture with efficient data mapping and reuse microarchitecture. Experiments demonstrate that our design can achieve up to $9.55\times$ and $69.75\times$ speedup over state-of-the-art NeRF accelerators and Xavier NX GPU in graphics rendering tasks with only $0.1$ PSNR loss.