ViSNeRF: Efficient Multidimensional Neural Radiance Field Representation for Visualization Synthesis of Dynamic Volumetric Scenes

TL;DR

ViSNeRF introduces a 3D-aware neural radiance field method that efficiently synthesizes visualizations of dynamic volumetric scenes from limited data, outperforming existing approaches in scientific visualization.

Contribution

The paper presents ViSNeRF, a novel multidimensional neural radiance field approach that reduces training data requirements and improves visualization synthesis for dynamic volumetric scenes.

Findings

Outperforms baseline methods in qualitative and quantitative evaluations

Requires fewer training images for accurate visualization reconstruction

Enables flexible exploration of visualization parameters

Abstract

Domain scientists often face I/O and storage challenges when keeping raw data from large-scale simulations. Saving visualization images, albeit practical, is limited to preselected viewpoints, transfer functions, and simulation parameters. Recent advances in scientific visualization leverage deep learning techniques for visualization synthesis by offering effective ways to infer unseen visualizations when only image samples are given during training. However, due to the lack of 3D geometry awareness, existing methods typically require many training images and significant learning time to generate novel visualizations faithfully. To address these limitations, we propose ViSNeRF, a novel 3D-aware approach for visualization synthesis using neural radiance fields. Leveraging a multidimensional radiance field representation, ViSNeRF efficiently reconstructs visualizations of dynamic volumetric scenes from a sparse set of labeled image samples with flexible parameter exploration over transfer functions, isovalues, timesteps, or simulation parameters. Through qualitative and quantitative comparative evaluation, we demonstrate ViSNeRF's superior performance over several representative baseline methods, positioning it as the state-of-the-art solution. The code is available at https://github.com/JCBreath/ViSNeRF.