Monocular Depth Decomposition of Semi-Transparent Volume Renderings

TL;DR

This paper evaluates and extends monocular depth estimation networks for semi-transparent volume renderings, enabling layered scene representations from single images with applications in scientific visualization.

Contribution

It demonstrates the adaptation of monocular depth networks to semi-transparent volumetric data and introduces methods to extract layered scene information from single renderings.

Findings

Existing depth estimation approaches can be adapted for semi-transparent volumes.

Layered scene representations can be derived from single volume renderings.

Applications include scene re-composition and enhanced scientific visualization.

Abstract

Neural networks have shown great success in extracting geometric information from color images. Especially, monocular depth estimation networks are increasingly reliable in real-world scenes. In this work we investigate the applicability of such monocular depth estimation networks to semi-transparent volume rendered images. As depth is notoriously difficult to define in a volumetric scene without clearly defined surfaces, we consider different depth computations that have emerged in practice, and compare state-of-the-art monocular depth estimation approaches for these different interpretations during an evaluation considering different degrees of opacity in the renderings. Additionally, we investigate how these networks can be extended to further obtain color and opacity information, in order to create a layered representation of the scene based on a single color image. This layered representation consists of spatially separated semi-transparent intervals that composite to the original input rendering. In our experiments we show that existing approaches to monocular depth estimation can be adapted to perform well on semi-transparent volume renderings, which has several applications in the area of scientific visualization, like re-composition with additional objects and labels or additional shading.