ENTIRE: Learning-based Volume Rendering Time Prediction

TL;DR

ENTIRE is a deep learning approach that accurately predicts volume rendering times by encoding structural properties and rendering parameters, enabling faster rendering and adaptive optimization.

Contribution

We introduce ENTIRE, a novel deep learning model that predicts rendering times considering multiple factors, with high accuracy and adaptability across frameworks.

Findings

Achieves high prediction accuracy across CPU and GPU frameworks.

Enables dynamic parameter tuning for stable frame rates.

Fast inference allows real-time application.

Abstract

We introduce ENTIRE, a novel deep learning-based approach for fast and accurate volume rendering time prediction. Predicting rendering time is inherently challenging due to its dependence on multiple factors, including volume data characteristics, image resolution, camera configuration, and transfer function settings. Our method addresses this by first extracting a feature vector that encodes structural volume properties relevant to rendering performance. This feature vector is then integrated with additional rendering parameters, such as image resolution, camera setup, and transfer function settings, to produce the final prediction. We evaluate ENTIRE across multiple rendering frameworks (CPU- and GPU-based) and configurations (with and without single-scattering) on diverse datasets. The results demonstrate that our model achieves high prediction accuracy with fast inference speed and can be efficiently adapted to new scenarios by fine-tuning the pretrained model with few samples. Furthermore, we showcase ENTIRE's effectiveness in two case studies, where it enables dynamic parameter adaptation for stable frame rates and load balancing.