Deep Learning based Super-Resolution for Medical Volume Visualization with Direct Volume Rendering

TL;DR

This paper introduces a deep learning-based super-resolution method for medical volume visualization that enhances low-resolution renderings to high-resolution with improved temporal stability, reducing computational costs.

Contribution

It presents a novel learning-based approach combining color and supplementary features for high-fidelity upscaling in volume rendering, applicable beyond medical imaging.

Findings

Achieves high-quality upscaling of volume renderings

Improves temporal stability with reprojection techniques

Reduces computational load for high-resolution visualization

Abstract

Modern-day display systems demand high-quality rendering. However, rendering at higher resolution requires a large number of data samples and is computationally expensive. Recent advances in deep learning-based image and video super-resolution techniques motivate us to investigate such networks for high-fidelity upscaling of frames rendered at a lower resolution to a higher resolution. While our work focuses on super-resolution of medical volume visualization performed with direct volume rendering, it is also applicable for volume visualization with other rendering techniques. We propose a learning-based technique where our proposed system uses color information along with other supplementary features gathered from our volume renderer to learn efficient upscaling of a low-resolution rendering to a higher-resolution space. Furthermore, to improve temporal stability, we also implement the temporal reprojection technique for accumulating history samples in volumetric rendering.