CNNs based Viewpoint Estimation for Volume Visualization

TL;DR

This paper introduces a CNN-based method for estimating viewpoints in volume visualization images, aiding understanding of user preferences and improving viewpoint selection through learned features and similarity measures.

Contribution

It presents a CNN-based viewpoint estimation approach with an overfit-resistant rendering pipeline and a similarity voting method for better viewpoint recommendation.

Findings

Achieves accurate viewpoint estimation across different rendering settings

Demonstrates effective recovery of viewpoints in published visualization images

Provides a semantic similarity measure for optimal viewpoint suggestion

Abstract

Viewpoint estimation from 2D rendered images is helpful in understanding how users select viewpoints for volume visualization and guiding users to select better viewpoints based on previous visualizations. In this paper, we propose a viewpoint estimation method based on Convolutional Neural Networks (CNNs) for volume visualization. We first design an overfit-resistant image rendering pipeline to generate the training images with accurate viewpoint annotations, and then train a category-specific viewpoint classification network to estimate the viewpoint for the given rendered image. Our method can achieve good performance on images rendered with different transfer functions and rendering parameters in several categories. We apply our model to recover the viewpoints of the rendered images in publications, and show how experts look at volumes. We also introduce a CNN feature-based image similarity measure for similarity voting based viewpoint selection, which can suggest semantically meaningful optimal viewpoints for different volumes and transfer functions.