Text-based Transfer Function Design for Semantic Volume Rendering

TL;DR

This paper introduces a novel language-guided method for designing transfer functions in volume rendering, using differentiable rendering and image-based loss to align visual output with user intent, simplifying the process.

Contribution

It presents a new approach that leverages language-vision models and differentiable rendering to create transfer functions from simple descriptions, reducing complexity and enhancing accessibility.

Findings

Effective transfer functions generated from text descriptions

Improved alignment between visual output and user intent

Streamlined transfer function design process

Abstract

Transfer function design is crucial in volume rendering, as it directly influences the visual representation and interpretation of volumetric data. However, creating effective transfer functions that align with users' visual objectives is often challenging due to the complex parameter space and the semantic gap between transfer function values and features of interest within the volume. In this work, we propose a novel approach that leverages recent advancements in language-vision models to bridge this semantic gap. By employing a fully differentiable rendering pipeline and an image-based loss function guided by language descriptions, our method generates transfer functions that yield volume-rendered images closely matching the user's intent. We demonstrate the effectiveness of our approach in creating meaningful transfer functions from simple descriptions, empowering users to intuitively express their desired visual outcomes with minimal effort. This advancement streamlines the transfer function design process and makes volume rendering more accessible to a wider range of users.